Problems of Education in The 21st Century, Vol. 79, No. 2, 2021

Problems of Education in the 21st Century, Vol. 79, No.
2, 2021
ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)
PROBLEMS OF EDUCATION
IN THE 21st CENTURY
Scientia Socialis, Ltd., a member of the Lithuanian Publishers Association and a member of
Publishers International Linking Association, Inc. (Crossref) in cooperation with SMC „Scientia
Educologica“, Lithuania, The Associated Member of Lithuanian Scientific Society, European
Society for the History of Science (ESHS) and ICASE (International Council of Associations for
Science Education)
The articles appearing in this journal are indexed, listed, and abstracted in:
Emerging Sources Citation Index (ESCI) (Web of Information Matrix for the Analysis of Journals
Science | Clarivate Analytics) (MIAR)
ERIC Directory of Open Access Scholarly Resources
Index Copernicus (ROAD)
EBSCO: Education Research Complete Eurasian Scientific Journal Index
Education Research Index Database Coverage List Crossref
“Core” coverage ProQuest (ProQuest Central; ProQuest Education
Central & Eastern European Academic Source Journals)
Database Coverage List (EBSCO-CEEAS) Europub: Academic and Scholarly Research
JournalSeek Publication Center
Ulrich`s Periodicals Directory The Scientific Literature Database (Scilit)
Contemporary Science Association/AAP databases QOAM
Open Academic Journals Index (OAJI) DOI
Zeitschriften Datenbank (ZDB) ResearchGate
GIGA Information Centre (Electronic Journals Dimensions
Library) SOBIAD
CNKI (China National Knowledge Infrastructure) Scholar CEEOL (Central and Eastern European Online
Turkish Education Index Library)
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
190 Publisher
Scientia Socialis, Ltd. in cooperation with SMC „Scientia Educologica“, Lithuania, the Associated
Member of Lithuanian Scientific Society, the Association of Lithuanian Serials, European Society
for the History of Science (ESHS) and International Council of Associations for Science Education
(ICASE)
Editors-in-Chief
Dr., Prof. Agnaldo Arroio, University of Sao Paulo, Brazil
Dr., Prof. Vincentas Lamanauskas, Vilnius University & SMC „Scientia Educologica“, Republic
of Lithuania
Editorial Board
Dr., Prof. Boris Aberšek, University of Maribor, Slovenia
Dr., Prof. Saleh A. Alabdulkareem, King Saud University, Saudi Arabia
Dr. Monica Baptista, University of Lisbon, Portugal
Dr., Prof. Martin Bilek, Charles University, Czech Republic
Dr., Prof. Andris Broks, University of Latvia, Latvia
Dr., Prof. Marco Antonio Bueno Filho, Federal University of ABC, Brazil
Dr. Paolo Bussotti, University of Udine, Italy
Dr., Prof. Muammer Calik, Trabzon University, Turkey
Dr. Saša A. Horvat, University of Novi Sad, Republic of Serbia
Dr., Prof. Margaret Chan Kit Yok, MARA University of Technology (UiTM), Malaysia
Dr., Angela James, University of Kwazulu – Natal, South Africa
Dr., Prof. Vladimir S. Karapetyan, Armenian State Pedagogical University named after
Kh. Abovyan, Armenia
Dr., Prof. Kuo-Hung Huang, National Chiayi University, Taiwan
Dr. Milan Kubiatko, Jan Evangelista Purkyně University in Ústí nad Labem, Czech Republic
Dr., Prof. Miroslaw Kowalski, University of Zielona Gora, Poland
Dr., Prof. Lay Yoon Fah, University Malaysia Sabah, Malaysia
Dr. Todar T. Lakhvich, Belarusian State Medical University, Republic of Belarus
Dr. Solange W. Locatelli, Federal University of ABC, Brazil
Dr. Eleonora Melnik, Petrozavodsk State University, Republic of Karelia, Russia
Dr., Prof. Danuše Nezvalova, Palacky University, Czech Republic
Dr. Osman Pekel, Suleyman Demirel University, Turkey
Dr., Prof. Yuriy Pelekh, Rivne State University of Humanities, Ukraine
Dr., Prof. Raffaele Pisano, University of Lille, France
Dr., Prof. Katarzyna Potyrala, Pedagogical University of Cracow, Poland
Dr. Costin Pribeanu, Academy of Romanian Scientists, Romania
Dr. Agneta Simeonsdotter Svensson, University of Gothenburg, Sweden
Dr. Uladzimir K. Slabin, University of Oregon, USA
Dr. Laima Railienė, Scientific Methodical Centre „Scientia Educologica“, Republic of Lithuania
Dr., Prof. Borislav V. Toshev, Sofia University, Bulgaria
Dr., Prof. Milan Turčani, Constantine the Philosopher University, Slovakia
Dr., Prof. Nicos Valanides, Educational Robotics and Science Organization, Cyprus
Copyright of this issue is the property of Scientia Socialis, Ltd, Lithuania. By virtue of their appearance in this open
access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings
Index Copernicus (IC™ Value): 120.63 (2019); ICDS (Secondary Composite Index Broadcasting): 9.5 (2018).
CGIJ OAJI: 0.350 (2018). H Index: 5 (2019)
Problems of Education in the 21st Century is an international, periodical, peer reviewed scientific journal,
issued by the Scientia Socialis Ltd. in cooperation with SMC „Scientia Educologica“.
Address: Scientia Socialis, Ltd.
Donelaičio Street 29, LT-78115 Siauliai, Lithuania
E-mail: problemsofeducation@gmail.com
Phone: +370 687 95668
WEB page: http://www.scientiasocialis.lt/pec
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Contents 191
Editorial
DRAWINGS TO LEARN SCIENCE: SOME REFLECTIONS

Solange W. Locatelli .................................................................................................. 192
Articles
EXAMINATION OF SYSTEM OF TRANSITION TO UPPER-SECONDARY SCHOOLS

FROM LOWER-SECONDARY SCHOOLS IN TERMS OF STUDENT, CLASS,
AND SCHOOL LEVEL VARIABLES
Hasan Yücel Ertem .................................................................................................. 194
ORGANIZATIONAL COMMITMENT LEVELS OF PRESCHOOL TEACHERS

AND ADMINISTRATORS’ LEADERSHIP STYLES
Kamil Arif Kırkıç, Türkan Demir Balcı ...........................................................................207
THE EFFECT OF EXPERIENCE ON REDUCING THE OVERCONFIDENCE

EFFECT IN TEACHERS
Robert Krause ......................................................................................................... 220
STUDENTS’ PERCEPTIONS ON DISTANCE EDUCATION:

A CASE STUDY IN MOZAMBIQUE
Alzira Manuel, Domingos Buque, Rosário Quive ............................................................... 229
STUDENT PERCEPTIONS OF ECUADORIAN VIRTUAL PLATFORMS

DURING THE COVID-19 PANDEMIC
Magda Francisca Cejas Martínez, Mercedes Navarro Cejas, Gina Silvana Venegas Alvarez,
Carlos Enrique Proaño Rodríguez, Derling Jose Mendoza Velazco .............................................241
POSTGRADUATE SUPERVISION PRACTICES IN EDUCATION RESEARCH AND

THE CREATION OF OPPORTUNITIES FOR KNOWLEDGE SHARING
Patrick Ngulube ....................................................................................................... 255
THE IMPACT OF TEACHER PROFESSIONAL DIARY ON THEIR REFLECTIVE,

COMMUNICATIVE AND PROFESSIONAL SKILLS
Eka Tkavashvili ...................................................................................................... 273
REVIEW OF THE STATE OF METHODOLOGICAL TRENDS IN OPEN AND

DISTANCE LEARNING LITERATURE 2009-2018
Scholastica Chizoma Ukwoma, Patrick Ngulube ................................................................. 296
THE FLIPPED CLASSROOM: MEDIA HYPE OR EMPIRICALLY BASED EFFECTIVENESS?

Lars-Frederik Weiß, Gunnar Friege ................................................................................. 312
Information
INSTRUCTIONS FOR AUTHORS ............................................................................. 333

This is an open access article under the
Creative Commons Attribution 4.0
International License
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
192
DRAWINGS TO LEARN SCIENCE:
SOME REFLECTIONS
Solange W. Locatelli
Federal University of ABC, Brazil
E-mail: solange.locatelli@ufabc.edu.br
Teacher needs to reinvent himself or herself all the time, proposing activities in the
classroom that enable students to build and reconstruct knowledge. Particularly in science, this
knowledge is mediated through scientific models, often inaccessible to students’ understanding.
For the comprehension of a chemical reaction, for example, the student is invited to imagine
how the interactions between the particles would be, the atomic rearrangement, totally abstract
thought and based on models, often expressed by visualizations, that need to be constantly
constructed and reviewed by students. However, how to revise these abstract concepts? What
strategies could be used? The answer to these questions is complex, but we would like to
propose a reflection on the use of drawings, as a possibility, among so many existing ones.
We start from the premise that drawing can be used as a strategy in any age group, with
the appropriate adjustments in the strategy, and not only with small children, as one might
suppose. Ainsworth, Prain and Tytler (2011) defend the use of drawings for learning in science,
bringing some important points, which are drawing as a way to engage students in the process,
as well as a way to learn to represent, to reason and communicate in science, thus constituting
an important strategy for science learning. In addition to the reasons mentioned and which we
agree with, we would like to highlight the drawing as a possibility to rethink, to revise concepts.
Locatelli and Davidowitz (2021) conducted a research with graduates from a Brazilian public
university, in which they were proposed to study a chemical reaction between ions. The activity
was carried out before the beginning of the COVID-19 pandemic, that is, in person. The students
were asked to propose an explanatory model at the submicro level to explain the chemical
reaction between silver ions and chloride in aqueous solution that they had just observed. After
this stage, they had the task of comparing what they had done with a drawing presented by
the researchers, scientifically accepted. This strategy, called the metavisual strategy, allowed
students to revise their model, reconstructing some concepts that had not been previously
perceived or that had been built in an inadequate way. Metavisualization, metacognition in
relation to visualization (Gilbert, 2005) has been used as a strategy to review concepts that are
linked to pictorial representations, as pointed out in this study. Locatelli and Davidowitz (2021)
conclude that it was possible to observe the manifestation and self-regulation of the concepts
demonstrated by the students, an essential and important component of metacognition that
favors science learning, which shows good possibilities for the use of drawings as a strategy for
learning science, in this case, related to chemistry.
Now, reflecting a little more, considering the context of the COVID-19 pandemic
that is plaguing the world, we could ask whether strategies, in this perspective, could be
used remotely. Yes, there are also possibilities and limits for this purpose. Locatelli (2020)
conducted a research during the year 2020 (at the beginning of the pandemic isolation period),
also with undergraduate students from a Brazilian public university, in which students also
had to propose an explanatory model for the chemical reaction between ions. However, the
context and methodology were different, since students developed the activity entirely by
remote means, using various technological resources: Facebook, google forms, among others.
Although, it was also observed that the metavisual activity with the use of drawings proved to
have good possibilities of revising concepts, these can occur in a partial way, since the students
https://doi.org/10.33225/pec/21.79.192 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Solange W. LOCATELLI. Drawings to learn science: Some reflections
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
found some difficulties in chemistry that could be overcome (self-regulated) and others did 193
not, evidencing the importance of teacher mediation during the activity, as a possible factor
that influenced the development of the activity, in addition to the context itself, a very peculiar
social isolation. It is important to note that this showed both possibilities and limitations, and
the concepts can and should be taken up using a synchronous class (via meet) or even face-to-
face classes return.
In the two studies described, it was possible to observe the relevant points pointed out by
Ainsworth, Prain and Tytler (2011) about the importance of using drawings. The self-regulation
of concepts (metacognitive aspect) was also present in both research studies, but the most
evident difference was the mediation of the teacher that could be done on the same day, in the
face-to-face case, differently from mediation through remote means, which was performed some
time later in a synchronous class. Actually, this was not a problem (since it was solved later),
but a limitation and which helped to strongly highlight the importance of teacher mediation in
the students' learning process. In addition, as already mentioned, the different context in which
the research took place (before and during the pandemic) certainly influenced the performance
of students in some way, which we intend to discuss at another time.
Even in the face of the challenges described, drawing can be an excellent opportunity to
learn science, enabling the review of scientific concepts.
References
Ainsworth, S., Prain, V., & Tytler, R. (2011). Drawing to learn in science. Science, 333(6046), 1096–
1097. https://doi.org/10.1126/science.1204153
Gilbert, J. K. (2005). Visualization: A metacognitive skill in science and science education. In J. K. Gilbert
(Eds.), Visualization in science education (pp. 9-27). Springer.
Locatelli, S. W., & Davidowitz, B. (2021). Using metavisualization to revise an explanatory model
regarding a chemical reaction between ions. Chemistry Education Research and Practice, 1, 1-14.
https://doi.org/10.1039/D0RP00339E
Locatelli, S. W. (2020). Entender una reacción química entre iones a través de tecnologías digitales - límites y
posibilidades [Understanding a chemical reaction between ions through digital technologies - limits
and possibilities]. Paradigma - Revista del centro de Investigaciones Educacionales Paradigma,
41(2), 172-189. https://doi.org/10.37618/PARADIGMA.1011-2251.0.p172-189.id863
Received: March 14, 2021 Accepted: April 02, 2021
Cite as: Locatelli, S. W. (2021). Drawings to learn science: Some reflections. Problems of
Education in the 21st Century, 79(2), 192-193. https://doi.org/10.33225/pec/21.79.192
Solange W. Locatelli PhD, Department of Chemistry, Federal University of ABC, Av. dos
Estados, 5001 - Bangú, Santo André - SP, 09210-580, Brazil.
E-mail: solange.locatelli@ufabc.edu.br
ORCID: https://orcid.org/0000-0002-7639-6772
ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.192

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
194
EXAMINATION OF SYSTEM OF
TRANSITION TO UPPER-SECONDARY
SCHOOLS FROM LOWER-SECONDARY
SCHOOLS IN TERMS OF STUDENT,
CLASS, AND SCHOOL LEVEL VARIABLES
Hasan Yücel Ertem
Zonguldak Bülent Ecevit University, Turkey
E-mail: ertem@beun.edu.tr
Abstract
Central exams which are reflection of structure of education systems determine many schooling processes.
Especially, scores coming from these exams are indicators for student performance. Transition to upper-
secondary school from lower-secondary school is based on “System of Transition to Upper-Secondary
Schools” (LGS). The current study aimed to examine the relationship between scores on System of
Transition to High Schools (LGS) and student, classroom, and school level variables. The design of the
study is a correlational research. The random sample consisted of 731 students from 47 classrooms in the
15 lower-secondary schools. Secondary data coming from educational institutions via school information
form were analyzed by Hierarchical Linear Modelling (HLM). School success score as a student-level
variable, experience of a class teacher as a classroom-level variable, and a number of exam branch
teachers, and a number of teachers assigned in support and training courses as school-level variables
predicted significantly students’ performance in LGS. Considering the results of the current study,
activities for professional development of teachers is recommended urgently to increase performance of
the students.
Keywords: educational policy, HLM, student performance, teacher experience, transition to upper-
secondary school
Introduction
Educational policies of the countries determine their education systems such as that
all school stakeholders take their positions by considering these systems. Attendance of
the students, teaching activities of the teachers and parental involvement are shaped by the
education system. National education system in Turkey depends on two unique characteristics:
centralized structure and crowdedness of stakeholder. These realities find out some necessities
to administer and govern education in Turkey. To name a few, national exams to select students,
central procedures to assign teachers, and financial plans to build new schools are topics
frequently discussed by public and political actors in Turkey. In these respects, the current study
will evaluate national and central exam in student, classroom, and school levels so that it will
both serve to fill a gap in the literature and make contribution to educational policies in Turkey.
Research Problem
Educational policy was led by Ministry of National Education (MoNE) in Turkey.

“2023 Education Vision” which includes structural and procedural reforms in order to improve

Hasan Yücel ERTEM. Examination of system of transition to upper-secondary schools from lower-secondary schools in terms of
student, class, and school level variables
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
education system of Turkey in different fields was implemented within the coordination of 195
MONE and Presidency (MoNE, 2019a). One of the focal points of this reform attempts is to
provide an efficient transition from lower-secondary schools to upper-secondary schools.
Turkey has been trying different systems of transition to upper-secondary schools
for years. The system between 2000 and 2003 was named as Exam for Entrance to Upper-
Secondary Schools (LGS). That system changed in 2004 and was entitled as Selection and
Placement Exam for Secondary Education Institutions (OKS). When comes to 2008, name of
new system became Placement Test (SBS) which was firstly administered to all students of 6th,
7th, and 8th grades. After 2009, only 8 students entered this exam. In 2014, system was again
changed and was called as Transition from Primary Education to Secondary Education (TEOG)
between 2014 and 2017 (Atılgan, 2018). Recently, System of Transition to Upper-Secondary
Schools (LGS) has been implemented since 2018 and will happen for the fourth time in 2021
(MoNE, 2020a).
The newest system has brought together important changes. First of all, even if the
students do not enter the exam, they will be placed in a high school by considering residential
address and middle school performance. Further, the students participating in the exam were
placed in the upper-secondary schools which are achievement-oriented schools and have
limited quota so that academic achievement of the student in the exam will be determinant.
To illustrate, 1029555 students participated in the exam and 138993 of them were placed in an
upper-secondary school. Remaining ones were placed by considering residential address, school
degree point, and attendance rates. On the other hand, questions in the exam are selected from
branches of Mathematics, Turkish Language, Science, Foreign Language, Turkish Republic
Revolution History and Kemalism, and Culture of Religion and Knowledge of Ethics (MoNE;
2019b). To summarize, systems of transition to high school changed many times in the last 20
years. It can be implied that there has been either unsteadiness or pursuit of better one.
Research Focus
Transition to upper-level schooling is the topic which has not lost its value for years due
to centralized exams and young population. These exams occupy the agenda of the public and
are studied by many researchers. Transition to upper-secondary school, transition to college,
and transition to professional life are the most three popular systems including central exams. In
2020, 1.473.303 lower-secondary school students (MoNE, 2020b), 2.296.138 upper-secondary
school students (Centre for Assessment, Selection, and Placement, 2020), and about 1.500.000
university students or graduates (Demir, 2020) were involved respectively in exams of transition
to upper secondary school, transition to college, and transition to professional life.
Literature based on central exams may be classified into three groups. The first group
is related to studies concentrating on relationship between exam scores and other educational
variables. School culture (Yılmaz & Bülbül, 2017), social and psychological effects (Çetin &
Ünsal, 2018), reading skills (Tuzlukaya, 2019), equality of opportunities (Sarıer, 2010) are the
topics linked to achievement in central exams, to name a few. In the second group, there are
studies focusing on the opinions of stakeholders about the exams. Attitudes towards central
exams (Buldur & Acar, 2019), economical burdens (Dinç et al., 2014), and system change
recommendation based on inclusion of other proficiencies (Karataş & Güleş, 2013) can be
shown as examples to studies viewing opinions of stakeholders. The third group of studies
concentrates on principles of assessment and evaluation. Item analysis based on mathematical
proficiency (Dönmez & Dede, 2020), knowledge levels of questions (Özden et al., 2014),
and item bias analysis (Bakan-Kalaycıoğlu & Kelecioğlu, 2011) were the studies related to
assessment and evaluation procedures.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
196 System of Transition to High School is the first central exam which Turkish students face
in the 8th grade in middle school. In addition to desire for enrolling in a high-quality school,
first experiences on central exam make this period significant. Therefore, researchers have
examined systems of transition to high schools for years. Because the most important output
of these exams is student performance or academic achievement, literature focuses mostly on
antecedents of achievement. Doğan (2015) examined TEOG score and school success and
found a positive relationship between grade point average and TEOG performance. Further,
individual features have an impact on success in central exams. Bal (2011) examined predictors
of achievement in the Placement Test and found that individual characteristics including self-
confidence, ambition, and attention and motivation predicted mostly success in Placement
Test. Similarly, study by Deniz and Kelecioğlu (2005) showed a positive relationship between
academic achievement in primary education and OKS scores.
In the literature of central exams, socio-economic and socio-cultural dynamics draw
attention. Köse (2007) examined the relation between academic achievement in Exam for
Entrance to High Schools and socioeconomic indicators and found that there was a relationship
between academic achievement and socioeconomic indicators such as private courses. Similar
study was conducted by Yavuz et al. (2016), who found that achievement was different by
socioeconomic differences among the schools. Aslan (2017) conducted a study to find out
determinants of student success in TEOG and found that educational levels of parents, house
income, and annual education expenditure for child predicted TEOG score of the student. Metin
(2013) examined factors affecting SBS and detected effects of grade level of students, education
level of parents, and home income on SBS performance.
The other side of the literature was commonly based on views, expectations, and
recommendations of school stakeholders. Akbaba-Altun and Çakan (2008) analyzed views of
school principals of successful schools in terms of LGS success. The authors found that practice
exams in the whole city, qualified educators, infrastructure, coordination among schools and
private courses, supporting schools, and pre-schooling were important in increasing LGS
success. The study by Şad and Şahiner (2016) focused on opinions of students, teachers, and
parents about TEOG and showed that stronger sides of exam were right to enter exam in own
school building, a make-up exam possibility, extending exam for two semesters, and not using
correction-for-guessing formula whereas the criticized points were unsecure exam conditions,
and promoting private course institutions and tutoring. A metaphor study was performed by Baş
and Kıvılcım (2019) to check perceptions of students about central exams like SBS and TEOG.
The authors found metaphors which can be grouped as “concept as anxiety, torture, disturbing
thing”, “concept as unfair, unnecessary race”, “concept as unconventional, irrelevant, variable,
or always existing thing”, “concept as turning point of our lives”, “concept of selective and
determinative of life”, and “concept as long or a short-term race”. Atılgan (2018) reviewed
both national and international implementations on transitions among educational levels and
recommended a model based on proficiency exam, combination of academic achievement,
guidance, and proficiency exam score, threshold value for either technical high school or right
to enter selection exam, and placement according to the achievement situation. To conclude,
systems for transition to upper education level have both strong and weak sides so that
researchers, scholars, educators, and policymakers have an intention to improve these systems.
Theoretical Framework
The theory of the current study was framed by Bronfenbrenner’s Bio-Ecological Theory.
Bronfenbrenner (1977) explained the core idea behind theory as the child’s interaction with
outside. According to the theory, student is placed in the center and this “outside” consisted of
five layers: microsystem, mesosystem, exosystem, macrosystem, and chronosystem. Child’s

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
interaction with closer elements around it, interaction between closer elements around child, 197
indirect environment around the child, social and cultural values, and changes over time were
the conceptualization of each layer, respectively. Components constructing purpose of the study
were sprinkled among these layers.
Research Aim and Research Questions
The latest system for transition to upper-secondary school in Turkey has been named as
“System of Transition to Upper-Secondary Schools” (LGS). The current study has aimed to
examine relation between scores on System of Transition to Upper-Secondary Schools (LGS)
and student, classroom, and school level variables. The research question was as below:
How well do student-level (school success score and year-end success score), classroom-
level (branch of class teacher and experience of class teacher), and school-level (number of
exam branch teachers, number of teachers assigned in support and training courses, ratio of
teacher to student, and non-attendance rate) variables predict LGS scores?
Research Methodology
General Background
The design of the current study was correlational research analyzing secondary data.
A correlational study has a purpose to explore relations among variables (Gall, Gall, & Borg,
2013). In correlational studies, there are mainly two types of variables which are predictor
and criterion variables. A predictor variable is the variable that is used to make a forecast
about criterion or outcome variable while a criterion variable is the variable being predicted
by a predictor variable (Creswell, 2012). Predictor variables were grouped as student-level
(school success score and year-end success score), classroom-level (branch of class teacher
and experience of class teacher), and school-level (number of exam branch teachers, number of
teachers assigned in support and training courses, ratio of teacher to student, and non-attendance
rate) variables. A criterion variable was the score coming from central LGS exam. The reasons
why these variables were selected were related to nature of the current study and access to data.
Nature of the current study is based on the dynamics of LGS such as that questions in LGS are
asked by considering curriculum of six branches. Further, students are trained by Support and
Training Courses. On the other side, data coming from schools and district national education
directorate were limited to these variables.
Sample
Sample consisted of the 731 eighth-grade students in an Anatolian Province. Moreover,

47 classrooms and 15 middle schools were included in the analysis. Random sampling was
used such as that 15 middle schools among totally 30 lower-secondary schools were selected
randomly to generalize results for the whole province including 30 schools. All of the classes
and students in these schools were used in the study. Two of these schools were private schools
while there were 13 public schools. Branches of 47 eighth-grade teachers were distributed as
Turkish (n= 12), Mathematics (n = 9), Life Science (n =8), Social Science (n = 8), Foreign
Language (n = 6), Sports (n = 2), Religious Science (n = 2), and Fine Arts (n = 1). Average
experience of teachers was 13.25 while average class size was 26.06. Average of year-end
success score, school success score, and LGS score of the students was respectively, 83.84/100,
84.22/100, and 312.59/500.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
198 Instrument and Procedures
Secondary data were procured from schools and district national education directorate.
In the current study, secondary data were requested via school information form. This form
included statements related to students, classrooms, and schools. Considering students, year-
end success score, school success score, and LGS score of the students were collected. Branches
and experiences of the eighth-grade teachers were classroom-level variables. Lastly, school-
level variables were the number of exam branch teachers, the number of teachers assigned in
support and training courses, the ratio of the number of teachers to the number of students, and
non-attendance rate. Table 1 depicts information about variables.
Table 1
Variables and Information about the Items
Variables Item Type

Dependent variable
LGS score Ratio scale
The first-level: Student-level variables
Year-end success score Ratio scale
School success score Ratio scale
The second-level: Classroom-level variables
Branch of class teacher Nominal
Experience of class teacher Ratio scale
The third level: School-level variables
Number of exam branch teacher Ratio scale
Number of teachers assigned in DYK Ratio scale
Rate of # of teacher to # of student Ratio scale
Non-attendance rate Ratio scale
Data Analysis

Secondary data were analyzed by Hierarchical Linear Modeling (HLM) such as that
analysis was conducted in student, class, and school levels. HLM is a complex form of least
squares method. In this method, independent variables are separated to levels hierarchically
so that variations on dependent variables are calculated more accurate. The reason why HLM
was used instead of Multiple Linear Regression was that student data were nested in both
classes and schools. School, class, and student characteristics must be examined within HLM
since education systems have a hierarchical structure. Further, HLM eliminates violation
of assumption of independent observation for nested data. Assumptions of HLM which are
linearity, normality, and homoscedasticity were checked and met. In the analysis, data collected
via school information form was entered in SPSS file. HLM was constructed to reveal within-
class, between-class / within school, and across-schools. In other words, data included three
levels: student as level-1, classroom as level-2, and school as level-3. In the analysis, student
version of HLM 8.0 was performed.

Research Results
Unconditional (Null) Model
In order to investigate how much variance in LGS score was caused by differentiation
in student-level, class-level, and school-level factors, unconditional model was performed. The

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
model is significant (t = 31.62, p < .001) such as that usage of HLM instead of regression is 199
significant as Table 2 demonstrates. In-class correlation coefficients (ICC) for student, class,
and school were calculated respectively as .74, .08, and .18 such as that 74% of variance in
LGS score was accounted for by student differences while 18% of variance in LGS score was
accounted for by differences across schools. Differences in variance also showed that running
HLM for this dataset was appropriate.
Table 2
Statistics about Null Model
Coefficient SE t ratio df
Intercept 304.70 9.63 31.62* 12

*p < .001
Level-1 Model (Student Level)
Level-1 model focusing on the effects of student variables on LGS performance revealed
that in-group variance decreased to 811.72 from 3838 when student variables added to the
model. Further, there was a decline in the amount of deviance from 7612.93 to 6554.92. Test
of variance-covariance components in the level-1 model was significant, Χ2 (12) = 240.63, p
< .001. All of these indicators showed that level-1 model was significant according to Garson
(2013).
Table 3 summarizes statistics related to level-1 model. Only school success score
predicted significantly LGS score. School success score was positively related to LGS score
such as that one-unit increase in school success score would increase LGS score 5.44 points.
However, year-end success score did not predict LGS score significantly.
Table 3
Level-1 Model
Coefficient SE t ratio
Intercept -184.20 11.83 -15.56*
Year-end success score .47 .31 1.51
School success score 5.44 .33 16.39*
*p < .001
Level-2 Model (Classroom Level)
Level-2 model focusing on the effects of classroom variables on LGS performance

depicted that in-group variance decreased to 8.15 from 403.90 due to addition of classroom
variables into the model. Moreover, there was a decline in the amount of deviance from 7612.93
to 6547.74. Test of variance-covariance components in the level-2 model was significant, Χ2
(12) = 398.45, p < .001. All of these indicators showed that level-2 model was significant
(Garson, 2013).
Table 4 presents statistics related to level-2 model. Only experience of class teacher
predicted significantly LGS score. Teacher experience was positively related to LGS score such
as that one-unit increase in experience of teacher would increase LGS score .89 points. On the
other hand, a branch of teacher did not make a significant difference on LGS score.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
200 Table 4
Level-2 Model
Intercept -191.91 11.92 -16.11*
Experience of class teacher .89 .28 3.12**
Branch of class teacher -.60 .66 -.91
*p < .001
**p < .004
Level-3 Model (School Level)
Level-3 model focusing on the effects of variables on LGS performance depicted that in-
group variance decreased to 309.76 from 932.26 due to the addition of classroom variables into
the model. Moreover, there was a decline in the amount of deviance from 7612.93 to 6540.53.
Test of variance-covariance components in the level-3 model was significant, χ2 (8) = 183.26, p
< .001. All of these indicators showed the significance of level-3 model as Garson (2013) stated.
Table 5 offered statistics related to level-3 model. Number of exam branch teachers
and number of teachers assigned in DYK predicted significantly LGS score. Number of exam
branch teachers were positively related to LGS score such as that one-unit increase in the
number of exam branch teachers would increase LGS score in 2.78 points. On the other side,
number of teachers assigned in DYK was negatively related to LGS score. One-unit increase
in the number of teachers assigned in DYK would cause 2.93 points decline in LGS score.
Non-attendance rate and the rate of number of teachers to number of students did not explain a
significant variance in LGS score.
Table 5
Level-3 Model
Intercept -228.08 25.42 -8.94*
Number of exam branch teacher 2.78 1.06 2.63**
Number of teachers assigned in DYK -2.93 1.19 -2.46**
Non-attendance rate 6.86 3.16 2.18
Rate of # of teachers to # of students .42 1.77 .24
*p < .001
**p < .05
Discussion
The current study examined relation between LGS performance and variables in the
levels of student, classroom, and school. Most of the variance in LGS score was explained by
differences in students. Further, the study showed that school success score as a student-level
variable, experience of class teacher as a classroom-level variable, and the number of exam
branch teachers and the number of teachers assigned in support and training courses as school-
level variables predicted significantly LGS achievement.
To begin, student differences explained more variance in LGS score than both school
and classroom differences did. This result remarks two important conclusions. The first one

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
is related to nature of the exam such as that the students who have a desire to be enrolled in 201
quality schools may enter the exam whereas the students who consent to be enrolled in the
school closest to home do not have to enter exam. Therefore, this situation may push student
characteristics to forefront. The second one is based on methodological choice such as that
schools were selected only from one district. Thus, equality of opportunities among schools or
something else is not a concern for the context of the study. On the other hand, literature has
studies showing large school differences (Aksu et al., 2017; Çelen et al., 2011, Mercik, 2015).
However, the common characteristics of these research studies was studying samples from
many kinds of schools. As a result, catching larger variations for school differences is quite
normal in this type of studies.
The current study performed three-level analysis based on HLM. The first level model
showed that one student-level variable predicted significantly LGS performance. Increase in
school success score would increase LGS performance whereas year-end score did not make a
significant difference on LGS performance. This differentiation remarks an important conclusion.
Year-end score is success score got at the end of a year. For the current study, year-end score is
the success score of the eighth grade. On the other hand, school success score is the average of
success scores of all grades. To conclude, long-term success was found more dominant on LGS
performance than short-term success. This finding is consistent with the idea in the literature.
Karakoç and Köse (2018) analyzed the relation between academic achievement in courses and
points of those courses in TEOG. The authors concluded that academic achievement was linked
to TEOG performance. Similarly, study by Kömürcü (2018) showed that there was a positive
relationship between high school success score and exam of entrance to higher education
institutions (YGS). Kablan (2010) examined the relation between Grade Point Average (GPA)
in Undergraduate and student performance on Civil Servant Selection Exam (KPSS) and found
that GPA predicted KPSS performance. In summary, long-term academic achievement may
determine performance in central exams.
The second level model showed that one classroom-level variable predicted significantly
LGS performance. Increase in experience of class teachers would increase LGS performance
while branch of the class teacher did not make a significant difference on LGS performance.
Experience of teacher was studied by many scholars such as that academic achievement of
students was linked to teacher experience. In this respect, results of the current study are
consistent with the literature (Abazaoğlu & Taşar, 2016; Atar, 2014; Bhai & Horoi, 2019). On
the other hand, a significant relation between branch of teacher and LGS performance was not
detected although the researcher of the current study expected. The reason why the researcher
had an expectation may be curiosity about the fact that the teacher whose branch is asked in
the LGS exam might have a positive impact on students’ preparation for the exam. However,
the current study did not find a significant result so that the study was included in the literature
which was not able to detect a significant difference on any educational variable for branch of
teacher (Akçay & Pekel, 2017; Çam & Üstün, 2016; Şanlı, 2017).
Considering school-level variables, the third-level model depicted that number of exam
branch teachers and number of teachers assigned in DYK as school-level variables predicted
LGS performance. Increase in the number of exam branch teacher would increase LGS
performance whereas increase in the number of teachers assigned in DYK would decrease LGS
performance. This finding offers an interesting dilemma for the literature. How similar variables
may form different results is remarkable. Primarily, literature has a common conclusion such
as that increase in the number of teachers would increase positive school outcomes (Dinç et al.,
2014; Doğan, 2004; Erdoğan et al., 2011; Pınar & Sarıbaş, 2009; Uzun & Çokluk-Bökeoğlu,
2019; Yıldırım, 2012). From this view, the current study is parallel to other studies in the
literature. On the other hand, finding related to the number of teachers assigned in DYK is
opposite to this common trend in the literature. The reason of this situation may be based on the

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
202 structure and process of DYK. In the literature, there are studies depicting problems observed in
the DYK. Tiredness due to the course time at the end of the day (Timur et al., 2020), higher non-
attendance rates (Bozbayındır & Kara, 2017), operation and planning problems (Canlı, 2019),
and lower motivation of the students (Topçu & Ersoy, 2019) are the problems in DYK, to name
a few. Further, Topçu and Ersoy (2019) emphasized that teachers gave lectures in DYK because
of additional income and bonus service points. To summarize, problems observed in DYK and
external motivators of teachers might have affected the results of this study.
Conclusions
The current study revealed out important conclusions on transition to upper-level schools
from lower-level schools. Evaluation of educational systems for school transition may be done
with multi-dimensional perspectives. In this respect, the current study performed the lenses of
students, classes, and schools. However, studies in the literature focused on only one dimension
of transition to upper-level schools. Thus, the current study showed a valuable attempt to close
a gap in the literature. In transition to upper-secondary school from lower-secondary school in
Turkey, perpetuity in school performance of students, experience of teachers, and a number of
teachers determined academic achievement of the students in central exams. Further, the study
underlined the importance of quality and content of the support rather than presence of support
since additional courses provided by government may not increase academic achievement of
the students. In spite of contribution of the current study to the literature, a remaining gap
has been still waiting to be filled. Relation of socio-cultural background of the students with
academic achievement is worth exploring.
Implications, Limitations, and Future Directions
Implications of the current study may be offered in terms of research, theory, and
practice. Considering research, the researcher of the current study did not come up with a study
investigating LGS that is a recent system on transition to high school in Turkey. Even though,
there were descriptive reports of MoNE presenting results and a qualitative study based on
the opinions of the teacher, literature lacking inferential study. Thus, the current study may be
evaluated as an attempt to fill a gap in the literature. Further, multi-level analysis may make
a contribution to the literature related to student performance. In terms of theory, findings
of the current study may serve the ideas behind the layers of bio-ecological theory. Student
performance is affected by both micro level factors and macro level factors. Considering micro
level factors, teachers as elements of microsystem and mesosystems were found affective in
student performance. In terms of macro level factors, centralized structure of LGS exam and
DYK as a policy may be the actors of exosystem. To summarize, the current study may relate
the perspective of the theory to the system on educational transition. Lastly, in practice, the
current study may give an opportunity to policymakers, educational leaders, and practitioners
to support students by enriching school settings. In this way, school effectiveness and school
improvement may be offered for future.
The researcher conducted the current study by considering some limitations. Although
the study had generalizability for population due to random sampling, the study was lack of
ecological generalizability since data were limited to only one district. Also, data of the current
study was collected via school information form. Analyses of the study were performed with
secondary data. In other words, the study was limited to variables and accessible secondary
data. Finally, the study had some delimitations such as that results of the current study were
limited to 2019 LGS scores, students living in an Anatolian province, and perspectives of bio-
ecological theory.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
The researcher has recommendations in terms of research, theory, and practice. First of 203
all, because the study is eventually based on secondary and quantitative data analysis, studies in
the future could conduct empirical studies including both quantitative and qualitative primary
or first-hand data from larger samples from different districts. Different variables for each level
should be conducted in the future studies. To illustrate, students’ psychological and emotional
characteristics as student-level, class size and parent meetings as classroom-level, and school
climate and school size as school-level could be included in the studies. In terms of theory,
the current study was concentrated on the first three layers of the study. Therefore, future
researchers may study macrosystem by integrating cultural issues. Further, the studies in the
future may compare the results of exams in two or more different years so that there will be a
link from student performance to the idea behind the chronosystem. Practical recommendations
to practitioners, educational administrators, and policymakers are related to the improvement
of school settings. Increasing student performance depends on thinking dynamics at each
level. Sustaining academic achievement of students for all grades, increasing affectivity of
parent-teacher associations, providing environments for professional development of teachers,
assigning more experienced 8th grade teachers, adjusting policies for appointing more teachers
at schools, and improving functionality of DYK through planning are recommended.
References
Abazaoğlu, İ., & Taşar, M. F. (2016). Relations of characteristics of science teachers and students with
the student achievement in science: A case analysis according to TIMSS 2011 Data. Elementary
Education Online, 15(3), 922-945.
Akçay, S., & Pekel, F. O. (2017). Investigation of prospective teachers' environmental awareness and
sensitivity in terms of different variables. Elementary Education Online, 16(3), 1174-1184.
http://dx.doi.org/10.17051/ilkonline.2017.330249
Aksu, G., Güzeller, C. O., & Eser, M. T. (2017). Analysis of maths literacy performances of students
with hierarchical linear modelling (HLM): The case of PISA 2012 Turkey. Education and
Science, 42(191), 247-266. https://doi.org/10.15390/EB.2017.6956
Altun, S. A., & Çakan, M. (2008). Factors affecting student success on exams: The case of successful
cities on LGS/ÖSS exams. Ilkogretim Online, 7(1), 157-173.
Aslan, G. (2017). Determinants of student successes in transition from basic education to secondary
education (TEOG) examination: An analysis related to non-school variables. Education and
Science, 42(190), 211-236. http://dx.doi.org/10.15390/EB.2017.6371
Atar, H. Y. (2014). Multilevel effects of teacher characteristics on TIMMS 2011 science
achievement. Education and Science, 39(172), 121-137. https://search.proquest.com/openview/3
af05d5e429cf7a6c92da50ea87ffaf1/1?pq-origsite=gscholar&cbl=1056401
Atılgan, H. (2018). Türkiye’de kademeler arası geçiş: dünü-bugünü ve bir model önerisi [Transitions to
schools in Turkey: Past, present, and model recommendation]. Ege Eğitim Dergisi, 19(1), 1-18.
https://doi.org/10.12984/egeefd.363268
Bakan-Kalaycıoğlu, D., & Kelecioğlu, H. (2011). Item bias analysis of the university entrance
examination. Education and Science, 36(161), 3-13.
Bal, Ö. (2011). Seviye belirleme sınavı (SBS) başarısında etkili olduğu düşünülen faktörlerin sıralama
yargıları kanunuyla ölçeklenmesi [Scaling factors effective in exam of success of placement by
considering the law of rank order judgment]. Eğitimde ve Psikolojide Ölçme ve Değerlendirme
Dergisi, 2(2), 200-209.
Baş, G. & Kıvılcım, Z. S. (2019). Türkiye’de Öğrencilerin Merkezi Sistem Sınavları ile İlgili Algıları:
Bir Metafor Analizi Çalışması [Perceptions of students on central system examinations in
Turkey: A metaphor analysis study]. Journal of Qualitative research in Education, 7(2), 639-
667. https://doi.org/10.14689/issn.2148-2624.1.7c.2s.8m
Bhai, M., & Horoi, I. (2019). Teacher characteristics and academic achievement. Applied
Economics, 51(44), 4781-4799. https://doi.org/10.1080/00036846.2019.1597963

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
204 Bozbayındır, F., & Kara, M. (2017). Problems faced at supporting and training courses (STC) and solution
suggestions according to teacher opinions. Sakarya University Journal of Education, 7(2), 324-
349.
Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American
Psychologist, 32(7), 513-531.
Buldur, S., & Acar, M. (2019). Ortaokul öğretmenlerinin merkezi sınavlara yönelik görüşleri [Lower-
secondary school students’ views for central exams]. Kastamonu Eğitim Dergisi, 27(1), 319-330.
Canlı, S. (2019). Opinions of school administrators and teachers on the support and training courses.
Cumhuriyet International Journal of Education, 8(2), 479-501.
Center for Assesment, Selection, and Placement. (2020). 2020 Yükseköğretim Kurumları Sınavı sayısal
veriler [Statistics of exam of higher education institutions]. https://dokuman.osym.gov.tr/
pdfdokuman/2020/YKS/yks_sayisal_27072020.pdf
Creswell, J. W. (2012). Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research. Pearson.
Çam, E, & Üstün, A. (2016). Öğretmenlerin mesleki tutumları ile yaşam boyu öğrenme eğilimleri
arasındaki ilişki [The relation between professional attitudes of teachers and their lifelong learning
trends]. Hitit Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 9(1), 459-475.
Çelen, F. K., Çelik, A., & Seferoğlu, S. S. (2011). Turkish education system and PISA results. Paper
presented at XIII. Conference on Academic Informatics, İnönü University, Malatya.
Çetin, A., & Ünsal, S. (2018). Social, psychological effects of central examinations on teachers and
their reflections on teachers' curriculum implementations. Hacettepe University Journal of
Education, 34(2), 304-323.
Demir, Ö. (2020). KPSS’ye kaç kişi girecek? KPSS’ye rekor başvuru [How many people will enter
KPSS exam? High application]. https://www.yenisafak.com/kpssye-kac-kisi-girecek-kpssde-
rekor-basvuru-h-2566316
Deniz, K. Z., & Kelecioglu, H. (2005). The relations between primary education academic achievement
grades and “the exam of student selection and placement for high school” scores. Ankara
University, Journal of Faculty of Educational Sciences, 38(2), 127-143.
Dinç, E., Uzun, C., & Çoban, O. (2014). Eğitimde kademeler arası geçişle ilgili öğretmen görüşlerinin
incelenmesi [Examination of transition to schools from the lenses of teachers]. Uşak
Üniversitesi Sosyal Bilimler Dergisi, 7(3), 209-235. https://dergipark.org.tr/en/pub/usaksosbil/
issue/21657/232859
Dinç, E., Dere, İ., & Koluman, S. (2014). Kademeler arası geçiş uygulamalarına yönelik görüşler ve
deneyimler [Views and experiences related to implementations on transition to schools]. Adıyaman
Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 7(17), 397-423. https://doi.org/10.14520/adyusbd.761
Doğan, c. (2004). Sınıf öğretmenlerinin derslere ilişkin görüşleri ve tercih ettikleri öğretim yöntemleri:
İstanbul örneği [Opinions of primary school teachers about courses and teaching methods: İstanbul
sample]. Türk Eğitim Bilimleri Dergisi, 2(2), 193-203.
Doğan, E. (2015). Examination of the relation between TEOG score and school success in terms of
various variables. Journal of Education and Training Studies, 3(5), 113-121.
Dönmez, S. M. K., & Dede, Y. (2020). Analysis of mathematics questions in the exams for transition to
secondary education (2016, 2017 and 2018 years) in terms of mathematical proficiency. Başkent
University Journal of Education, 7(2), 363-374.
Erdoğan, İ., Meşeci-Giorgetti, F., & Çifçili, V. (2011). Seviye belirleme sınavı yordayıcıları ve eğitim
imkânlarında eşitlik sorunu [Predictors of the elementary school proficiency exams and issues of
equality in educational facilities]. Kuram ve Uygulamada Eğitim Bilimleri, 11(1), 215-228.
Gall, G., & Gall, J. P. Borg. (2003). Educational research: An introduction (7th ed.). Pearson.
Garson, G. D. (2013). Hierarchical linear modeling: Guide and applications. Sage.
Kablan, Z. (2010). Akademik mezuniyet ortalama puanı ile KPSS başarı puanı arasındaki ilişki [The
relation between grade point average of undergraduate and KPPS achievement]. Uldağ Üniversitesi
Eğitim Fakültesi Dergisi, 23(2), 451-470.
Karakoç, G., & Köse, İ. A. (2018). The relationship between elementary school academic achievement
levels and high school entrance exam scores. Cumhuriyet International Journal of Education, 7(2),
121-142.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Karataş, S., & Güleş, H. (2013). Öğretmen atamalarında esas alınan merkezi sınavın (KPSS) öğretmen 205
adaylarının görüşlerine göre değerlendirilmesi [Evaluation of KPPS exam for teacher assignment
in terms of teacher perspective]. Kuramsal Eğitimbilim Dergisi, 6(1), 102-119.
Kömürcü, İ. (2018). The predictive validity of the scores obtained in state conservatories’ student selection
exam on students’ academic achievement. Journal of Higher Education and Science, 8(2), 298-
304.
Köse, M. R. (2007). Aile sosyoekonomik ve demografik özellikleri ile okul ve özel dershanenin liselere giriş
sınavına katılan öğrencilerin akademik başarıları üzerindeki etkileri [Effects of socio-economic
and demographic features of families and school and private courses on academic achievement
of students in exam of entrance to upper secondary schools]. Eğitim Bilim Toplum, 5(17), 46-77.
Mercik, V. (2015). The relationship between the equality of opportunity in education, general social
success, and justice: The comparison of Finland and Turkey experiences within the scope of PISA
project. (Unpublished master's thesis). Balıkesir University, Balıkesir.
Ministery of National Education (MoNE). (2019a). 2023 Eğitim Vizyonu [2023 education vision].
http://2023vizyonu.meb.gov.tr/
Ministery of National Education (MoNE). (2019b). 2019 ortaöğretim kurumlarına ilişkin merkezi
sınavı [2019 central exam for upper-secondary schools]. https://www.meb.gov.tr/meb_iys_
dosyalar/2019_06/24094730_2019_Ortaogretim_Kurumlarina_Iliskin_Merkezi_Sinav.pdf
Ministery of National Education (MoNE). (2020a). LGS’de ek kapasite oluşturuldu [Additional capacity
for LGS exam]. https://www.meb.gov.tr/2020-lgsde-ek-kapasite-olusturuldu/haber/20330/tr
Ministery of National Education (MoNE). (2020b). 2020 ortaöğretim kurumlarına ilişkin merkezi
sınav [2020 central exam for upper-secondary schools]. http://www.meb.gov.tr/meb_iys_
dosyalar/2020_07/17104126_2020_Ortaogretim_Kurumlarina_Iliskin_Merkezi_Sinav.pdf
Özden, M., Akgün, A., Çinici, A., Sezer, B., Yıldız, S., & Taş, M. M. (2014). Merkezi sistem ortak sınav
fen bilimleri sorularının Webb’in bilgi derinliği seviyelerine göre analizi [Analysis of science
questions used in central system common exam according to Webb’s dok levels]. Adıyaman
Üniversitesi Fen Bilimleri Dergisi, 4(2), 91-108.
Sarier, Y. (2010). An evaluation of equal opportunities in education in the light of high school entrance
exams (OKS-SBS) and PISA results. Journal of Kirsehir Education Faculty, 11(3), 107-129.
Şad, S. N., & Şahiner, Y. K. (2016). Students' teachers' and parents' views about transition from basic
education to secondary education (BESE) system. Elementary Education Online, 15(1), 53-76.
Şanlı, Ö. (2017). Analyzing the perceived stress level of teachers with regards to some variables. Electronic
Journal of Social Sciences, 16(61), 385-396.
Timur, S., Kahraman, S., Timur, B., & İşseven, A. (2020). Destekleme ve Yetiştirme Kurslarına (DYK)
ilişkin ortaokul öğrencilerinin görüşleri [Opinions of lower-secondary school students on
Supporting and Training Courses (DYK)]. Trakya Eğitim Dergisi, 10(1), 216-228.
Topcu, Ü. İ., & Ersoy, Ü. M. (2019). Teachers’ views on the supportive and training courses in Turkish
education system. E-International Journal of Educational Research, 10(3), 61-75.
Tuzlukaya, S. (2019). 8th class Turkish lessons exam questions examined in regard to PISA reading
sufficiency skills. The Journal of International Lingual Social and Educational Sciences, 5(1),
92-100.
Uzun, G., & Çokluk-Bökeoğlu, Ö. (2019). Akademik başarının okul, aile ve öğrenci özellikleri
ile ilişkisinin çok düzeyli yapısal eşitlik modellemesi ile incelenmesi [Examination of
academic achievement with school, family, and student features by multi-level structural
equation modelling]. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 52(3), 655-
684. https://doi.org/10.30964/auebfd.525770
Yavuz, S., Odabaş, M., & Özdemir, A. (2016). Öğrencilerin sosyoekonomik düzeylerinin TEOG
matematik başarısına etkisi [Impact of socio-economic levels of students on math achievement
in TEOG exam]. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 7(1), 85-95.
https://doi.org/10.21031/epod.86531
Yıldırım, K. (2012). Basic factors determining success of educational quality in Turkey by PISA
2006. Journal of Turkish Educational Sciences, 10(2), 229-255.
Yılmaz, S., & Bülbül, T. (2017). Merkezi sınavların okul kültürüne yansımalarının değerlendirilmesi
[Evaluation of reflections of central exams on school culture]. Trakya Üniversitesi Eğitim
Fakültesi Dergisi, 7(2), 578-617. https://doi.org/10.24315/trkefd.315491

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
206
Received: February 24, 2021 Accepted: April 03, 2021
Cite as: Ertem, H. Y. (2021). Examination of system of transition to upper-secondary schools

from lower-secondary schools in terms of student, class, and school level variables. Problems
of Education in the 21st Century, 79(2), 194-206. https://doi.org/10.33225/pec/21.79.194
Hasan Yücel Ertem PhD in Educational Administration and Planning, Assistant Professor,
Zonguldak Bülent Ecevit University, Kepez District Prof. Muammer Aksoy
Street 126, 67100 Zonguldak, Turkey.
E-mail: ertem@beun.edu.tr
ORCID: https://orcid.org/0000-0001-9058-641X

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
ORGANIZATIONAL COMMITMENT 207
LEVELS OF PRESCHOOL TEACHERS AND

ADMINISTRATORS' LEADERSHIP STYLES
Kamil Arif Kırkıç
Istanbul Sabahattin Zaim University, Turkey
E-mail: kamil.kirkic@izu.edu.tr
Türkan Demir Balcı

Ministry of National Education, Turkey
E-mail: turkkan_28@hotmail.com
Abstract
The organizational commitment levels of preschool teachers are crucial to have an effective early childhood
education. One of the most influential factors affecting organizational commitment is administrators'
leadership style. This research explored the correlation between administrators' leadership styles and
organizational commitment levels of preschool teachers. The correlational survey design was used as
one of the quantitative research models in this study. Participants in the study were preschool teachers
in Istanbul's Küçükçekmece district. The research was conducted during the 2017-2018 academic year.
There are 237 teachers in the study sample. The 5-point Likert type Organizational Commitment Scale
with 27 items and the Leadership Style Scale with 35 items were used to collect data. It was found
that teachers generally identified with their organizations and internalized the organization’s values,
while school administrators were perceived as using a transformational leadership style. The study
found a statistically significant correlation between the leadership styles of administrators and the levels
of organizational commitment of preschool teachers. In preschool institutions, teachers' institutional
commitment is essential for the development of students. Therefore, administrators should be sensitive
to their leadership styles to make the commitment levels of preschool teachers high. The research has
limitations like being a local one, done in a relatively short time, and the participants were only teachers
working in public schools.
Keywords: leadership style, organizational commitment, preschool teacher, correlational survey,
educational leadership
Introduction
The most crucial responsibility in educational institutions that strive for perfection lies
with the institution managers to work effectively. The most significant difference between a
high-performance educational institution and a low-performance educational institution is the
institution managers’ effectiveness in these institutions. This suggests that it is the leadership
style demonstrated by institution managers that enables educational institutions to progress
effectively. These managers should plan and fulfill their work following accepted rules.
Managers of educational institutions play a crucial role in deciding educational quality. Their
leadership styles greatly influence managers' ability to unlock the school's full potential.
The organizational commitment of teachers and administrators is accepted as an indicator in
determining an educational institution's success (Banjarnahor et al., 2018).
For an educational organization to have committed employees, those in management
have significant responsibilities because the organization's success relies on its employees and

Kamil Arif KIRKIÇ, Türkan Demir BALCI. Organizational commitment levels of preschool teachers and administrators’ leadership
styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
208 is directly related to management’s level of influence and authorization. The more effective
the leader is in communicating the duties and objectives of the institution to the employees,
the more successful those people will be in adapting to the corporate atmosphere (Avcı, 2015,
p.88).
In an educational institution, administrators and teachers influence each other. Through
this interaction, school administrators must play an active role in achieving the institution's goals,
increasing teachers' affiliation to the institution, addressing the situations that teachers want,
and impacting them. From another perspective, organizational commitment means embracing
the organization's interests and success in all its effectiveness (İbicioğlu, 2000). Leadership
behavior was also stated as a factor in studies conducted to determine the factors affecting
organizational commitment (Çetin, 2004). According to some researchers, the behaviors and
practices of those acting as leaders affect employees' organizational commitment.
Research Focus
Employees show low commitment to an organization when they believe leaders are not
helpful (Zangora, 2001). Due to the essential connection between organizational commitment
and the leadership styles owned by administrators, different leadership styles should be
evaluated to examine the commitment levels of all employees. Leadership approaches cannot
be separated from the social and political characteristics of society. A successful leader
examines the social structure and acts following its social and cultural features. A leader should
consider the institution's diversity as part of its wealth and work to unite all aspects. A leader
must influence all people, have good communication skills, have a holistic approach, know the
strengths and weaknesses of employees, have a positive approach, be fair, and make decisions
guided by equality criteria (Kantos, 2011).
The importance of leadership has been highlighted and expressed in terms of the leadership
styles best suited to preparing the ground for achieving organizational goals and self-sacrificing
work within an institution (Abasilim et al., 2019). The leader and their employees influence
each other. When the leader's influence varies according to the individual characteristics of
the audience, one should look at the quality of the leader and the qualities of the employees.
While the leadership and management approach of the 19th Century was based on a structure of
tradition, new approaches to leadership have arisen, which are very different (Yıldırım, 2006).
Transformational leadership, sustaining leadership, and laissez-faire leadership are the three
types of leadership defined by researchers (Yalçın, 2014).
The transformational leader attaches importance to new approaches and focuses on
explaining the organization's mission. This type of leader attempts to remain loyal to his ideals
and create reliability, honesty, and respect. A transformational leader has high performance,
broad experience, and adequate working power, which differs from managerial behavior
based on tradition. They are also less confused about tasks. In short, transformational leaders
exhibit the behaviors, values, and ideals of a leader (Hoşcan et al., 2004). Transformational
leaders work on adding value to the community's goals beyond their own individual goals.
School administrators who exhibit transformational leadership behavior create trust, respect,
cooperation, high motivation, and loyalty among their employees (Gardner, 2018). Past studies
report that transformational leadership is more effective, productive, and innovative. Through
transformational leadership in educational institutions, an effort is made to satisfy employees,
to bring them together in a shared vision, and to act in line with common goals (Chiun Lo et
al., 2009).
In transactional leadership, employees are rewarded if they comply with the operational
standards, focus on achieving institutional goals, and complete their work. It is a form
of leadership that is supported by bureaucratic supervision and forms a legally determined

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
position of duty, stating that problems may arise if tasks are not completed (Özgener & Kılıç, 209
2009). A transactional leader enables employees to work by making positive use of their past
experiences or to make them more useful. Less attention is paid to employees being productive
and innovative. Besides hard and disciplined work, such leaders tend to be tolerant, wise, and
satisfied when work progresses according to the organizational plan and discipline (Eren, 2018).
The liberal leader offers endless rights to employees and prepares an environment for
them to act independently. These leaders refrain from giving feedback to their employees by
avoiding openly speaking up regarding events and situations that matter. Such leaders want to
maintain existing conditions in the institution (Akan et al., 2017). According to this approach, a
leader is the person who least deserves the right to lead. Leaders allow employees to work in line
with their strengths and exhibit behaviors that allow them to make their goals and arrangements
in line with their individual resources. In other words, leaders who allow complete freedom do
not claim rights and empower employees (Karip, 1998).
While it is accepted that the concept of organizational commitment defines the
relationship between the organization and its functions, discussion continues about the nature
of the concept. Organizational commitment, organizational psychology, and organizational
behavior are topics that are continually increasing in the literature. Both theoretical and
practical efforts have been taken in increasing interest in organizational commitment in order to
reveal results and expectations. The organizational commitment process helps recognize hidden
meanings for employees (Okçu, 2011). Organizational commitment is defined as the relative
strength of an individual's participation in a particular organization and how they identify with
the organization. Organizational commitment is about adopting organizational values, being
committed to the organization, and having the desire to participate in the work that needs to be
done. As seen in the literature, organizational commitment comprises at least three elements
(Çokluk & Yılmaz, 2010).
1. A strong belief in, and a recognition/acknowledgment of, the goals and values of the
organization,
2. Putting in significant effort for the organization,
3. Remaining as a member of the organization.
Organizational culture and organizational commitment are two integrated concepts.
Since organizational commitment is employees’ adoption of organizational culture, values,
and goals, the organizational culture is critical for those organizations to have a high level of
commitment. This suggests that by ensuring the desire of new members of an organization to
be a part of the organizational culture, its expectations will be met. Success is achieved when
organizational culture is accepted and adopted by employees (Erceylan, 2010).
Organizational commitment is classified as affective commitment, continuance
commitment, normative commitment (Allen &Meyer, 1990, p.3). These conceptual structures
refer to the link between the organization and the employee. Affective commitment, where
employees desire to remain in the organization, develops from their positive experiences.
Employees’ needs determine continuance commitment in terms of the social and economic cost
of leaving the organization. In addition, normative commitment is the employees’ feeling that
it is necessary to remain in the organization (Türkmen, 2016).
Compliance, recognition, and internalization are the three dimensions of organizational
engagement. Individuals have adopted the organization's goals and authority to receive
incentives or avoid punishment (O'Reily & Chatman, 1986, p. 493). Celep (2014, p. 16) has
stated that compliance in such situations occurs when employees go beyond the expectations
of others in order to achieve the awards, they deem personally necessary. Individuals behave
out of necessity, not because they believe in the organization. Harmony is always imperative.
Managers may use their authority to ensure that employees carry out their requests, so employees
must obey. However, such compliance is done reluctantly. According to Handy, this type of

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
210 adaptation brought about by necessity has been based on control rather than trust and eliminates
the individual's right to choose (cited in Balay, 2000, p.104). In schools where altruism and self-
sacrifice are expected, teachers do not feel a commitment to the organization, which negatively
affects the educational activities, quality, and effectiveness of schools (İmamoğlu, 2011, p.29).
The administrators and educators of institutions with high organizational commitment create
a reliable and successful working atmosphere, the execution of sustainable studies, the desired
level of individual and academic progress, and the training of self-confident and self-accepting
people. These administrators and educators increase their employees’ commitment to their
organizations, support their personal development through proper study, provide more support
and innovative studies to students, and allocate more time to the institution. These behaviors
also contribute to the success of the institution (Avcı, 2017, p.67).
Internalization, the final element of commitment, is the harmony of personal and
organizational values and the effect of organizational values on employees’ behavior. For
internalization, the individual must voluntarily accept and adopt organizational values as their
values, without elements of coercion. Therefore, internalization includes normative commitment
(Balcı, 2003). This type of commitment is most helpful to organizations because internalization
is independent of the origin of the source of influence. However, there is less chance of success,
and it takes a longer time to develop. When internalization occurs, it is not necessary to find new
resources to convince employees continually. With this commitment, employees accept the new
ideas, developments, and behavior as their own (Balay, 2000). According to Heidmets and Liik,
the leadership styles of administrators can be a factor in determining teachers' commitment to
school and their emotional attachment to the school (2014). In some research studies, it was
found that teacher commitment was linked explicitly to principals’ leadership behaviors (Da’as,
2020; Freeman & Fields, 2020; Griffioen & de Jong, 2017).
This study aimed to search the correlation between the leadership styles of administrators
in terms of the perceptions of preschool teachers and the levels of organizational commitment
of preschool teachers. The key research question was whether there was a correlation between
administrators' leadership styles according to preschool teachers' perceptions and preschool
teachers' organizational commitment levels. To this end, the study sought to find answers to the
following questions:
a) What is the level of teachers' organizational commitment?
b) According to teachers’ perceptions, what kind of leadership styles do school
administrators show?
c) Is there a significant correlation between the leadership styles of school administrators
and the organizational commitment of teachers?
d) Do leadership styles of administrators predict the organizational commitment of
preschool teachers?
General Background
The research was carried out by using the relational survey model in order to determine the
correlation between the levels of the organizational commitment of teachers and the leadership
styles of school administrators. Survey models attempt to explain an existing situation in the
past or present. The survey model is the arrangements made on the entire group or a sample
to reach a general judgment about many elements of the whole group (Karasar, 2017). The
research was conducted in the 2017-2018 academic year. The total duration for the research
was eight months.

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Sample 211
The population of the research were 596 teachers working in the independent public
kindergartens in the Küçükçekmece district of Istanbul and as well as in the kindergartens
within public primary, secondary, and high schools. The number of teachers to be sampled
was calculated to be at least 234 within a 95% confidence level and a 5% margin of error.
The calculation was done by using the Raosoft sample size calculator (Raosoft, 2018). The
sample consists of 237 teachers, and they were selected through a random sampling method
from among 596 teachers. Of the teachers in the sample group, 55 (23.2%) were in the 25 and
under-age group, 84 (35.4%) in the 26-30 age group, 54 (22.8%) in the 31-35 age group, 44
(18.6%) in the 36 and over age group. Of them, 145 teachers (61.2%) work in independent
kindergarten, and 92 (38.8%) in kindergarten within primary, secondary, and high schools. One
hundred and sixty-one of the teachers (67.9%) worked for 1-3 years at their current school, and
76 (32.1%) have worked for four years or more. Necessary permissions were obtained from
the Ministry of Education. Verbal consent was obtained from the participants. They were told
that they could stop answering the questionnaires and participate in the study if they were not
volunteers anymore.
Instrument and Procedures
The questionnaire to collect the research data includes the "Organizational Commitment
Scale", first developed by Balay (2000). The scale consists of three sub-dimensions,
compliance, identification, and internalization. It was prepared as a 5-point Likert type rating
from (1) Strongly Disagree to (5) Very Strongly Agree. The compliance sub-dimension of
the organizational commitment scale consists of items 1-8, the identification sub-dimension
contains items 9-16, and the internalization sub-dimension contains items 17-27. Analyses can
be made on factor scores for three factors in the scale, and a single score cannot be obtained
from the scale. In the study, the Cronbach's alpha reliability coefficient was calculated as .82
for the compliance sub-dimension, .90 for the identification sub-dimension, and .90 for the
internalization sub-dimension.
The second scale that the research employed was the "Leadership Style of School
Administrators Scale," developed by Akan, Yıldırım, and Yalçın (2014). The scale is prepared
as a 5-point Likert type scale rating from (1) Strongly Disagree to (5) Strongly Agree. The scale
consists of three sub-dimensions: transformational leadership style, transactional leadership
style, and laissez-faire leadership style. Cronbach's alpha reliability coefficient was .95 for
transformational leadership, .65 for transactional leadership, and .87 for laissez-faire leadership.
Data Analysis
The data obtained in the study was coded and loaded into the computer and analyzed
using the "SPSS 25.0 for Windows" statistical program. The skewness and kurtosis values were
used to determine whether the data had a normal distribution. The values were determined
as 1.00 and 1.15 in the "compliance" sub-dimension of the organizational commitment sub-
dimension, -.57 and .11 in the "identification" sub-dimension, -.40 and .76 in the "internalization"
sub-dimension, and -.63 and 1.25 in the whole organizational commitment scale. The
"transformational leadership" sub-dimension of the leadership styles of school administrators
was calculated as -.43 and .59, the " transactional leadership" sub-dimension as .04 and .35, and
the " laissez-faire leadership" sub-dimension as .64 and .60. These were accepted as normal
distribution because Tabachnick and Fidell (2013) accept the data distribution as normal when
the skewness and kurtosis values are within the range of ± 1.5. According to George and

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
212 Mallery (2010), the skewness and kurtosis values being within ± 2 intervals is acceptable for
normality. Based on these expressions, the distribution is accepted as normal.
Pearson Product Moment Correlation Analysis was used to determine the correlation
between preschool teachers' organizational commitment and school administrators' leadership
style. Regression analysis was conducted to determine to what extent the leadership styles of
school administrators affect teachers' organizational commitment.
Research Results
Leadership Styles of Administrators According to Preschool Teachers' Perceptions

And Organizational Commitment Levels of Preschool Teachers
As can be seen in Table 1, organizational commitment levels of preschool teachers are

primarily in the "internalization" sub-dimension, followed by the "identification" sub-dimension,
and the least in the "compliance" sub-dimension. Organizational commitment perceptions of
teachers are at the level of "absolutely disagree" in the "compliance" sub-dimension; "I agree"
in the "identification" sub-dimension; and "I agree" in the "internalization" sub-dimension.
Table 1
Arithmetic Average and Standard Deviation Values of the Organizational Commitment Scale
Sub-Dimensions
Dimensions M SD
Compliance 1.74 0.59
Identification 3.81 0.77
Internalization 3.89 0.60
Table 2 shows the Leadership Styles Scale Sub-Dimensions. As shown in Table 2,

according to the perceptions of the teachers, school administrators mostly have a "transformational
leadership" style (M = 4.08), then a "transactional leadership" style (M = 2.57), and the least
have a " laissez-faire leadership" style. (M = 1.93) according to the perceptions of the teachers.
At the "I accept" level, the school administrators displayed "transformational leadership"; at the
"indecisive" level, they displayed "transactional leadership"; and at the “disagree” level, they
displayed "laissez-faire leadership".
Table 2
Arithmetic Average and Standard Deviation Values of School Administrators' Leadership
Styles Scale Sub-Dimensions
Dimensions M SD
Transformational Leadership 4.08 0.59
Transactional Leadership 2.57 0.61
Laissez-Faire Leadership 1.93 0.68

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Evaluation of the Correlation between the Administrators’ Leadership Style and 213
Preschool Teachers’ Organizational Commitment
Table 3 shows the result of the Pearson correlation analysis conducted to determine
whether there is a statistically significant correlation between the sub-dimensions of the leadership
styles scale of school administrators and the sub-dimensions of the organizational commitment
scale. There was a statistically significant negative correlation between "transformational
leadership" and "compliance" (r = -.467; p <.01). There is a statistically significant positive
correlation between "transformational leadership" and "identification" (r = .630; p <.01). There
was a statistically significant positive correlation between "transformational leadership" and
"internalization" (r = .514; p <.01). There was a statistically significant positive correlation
between “transactional leadership" and "compliance" (r = .431; p <.01). There was a statistically
significant negative correlation between “transactional leadership" and "identification" (r =
-.385; p <.01). There was a statistically significant negative correlation between "transactional
leadership" and "internalization" (r= -.266; p <.01). There was a statistically significant
positive correlation between “laissez-faire leadership" and "compliance" (r= .487; p <.01).
There was a statistically significant negative correlation between "laissez-faire leadership" and
"identification" (r = -. 505; p <.01). There was a statistically significant negative correlation
between "laissez-faire leadership" and "internalization" (r = -. 399; p <.01).
Table 3
Pearson Correlation Analysis Results Conducted to Determine the Correlation between
Preschool Administrators' Leadership Styles and Teachers' Organizational Commitment
Dimensions Compliance Identification Internalization
r -.467 .630 .514

Transformational Leadership
p .0001 .0001 .0001
r .431 -.385 -.266

Sustainable Leadership
p .0001 .0001 .0001
r .487 -.505 -.399

Laissez-Faire Leadership
p .0001 .0001 .0001
Evaluation of School Administrators’ Leadership Styles Predicting

Preschool Teachers' Organizational Commitment
A regression analysis was conducted to explore the prediction level of school

administrators’ leadership styles on organizational commitment levels of preschool teachers.
The regression analysis in Table 4 reveals that transformational leadership style explains 22%
of the total variance of the compliance dimension (R2=.218; p <.001) and 26% of the total
variance of the internalization dimension (R2=.264; p <.001). Furthermore, the transformational
leadership style explains 40% of the identification dimension's total variance (R2=.397; p <.001).
Transactional leadership style explains 19% of the total variance of compliance dimension
of organizational commitment (R2=.186; p <.001), 15% of the total variance at the level of
identification dimension (R2 =.148; p <.001), and 7% of the total variance at the internalization
dimension level (R2 =.071; p <.001).

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
214 Laissez-Faire leadership style explains 24% of the total variance of compliance dimension
of organizational commitment (R2=.237; p <.001), 26% of the total variance of identification
dimension (R2=.255; p <.001), and 16% of the total variance at the internalization dimension
(R2=.159; p <.001).
Table 4
Regression Analysis Results Related to the Level of Leadership Styles of School Administrators
Predicting Teachers' Organizational Commitment
Dependent
Independent Variable B SE (β) R R² F p
Variable
Transformational Compliance -.47 .11 -.47 .467 .218 65.456 .001
Leadership
Identification .82 .07 .63 .630 .397 154.93 .001
Internalization .52 .06 .51 .514 .264 84.431 .001
Compliance .42 .06 .43 .431 .186 53.686 .001

Transactional
Leadership Identification -.49 .08 -39 .385 .148 49.887 .001
Internalization -.26 .06 -27 .266 .071 17.858 .001
Compliance .42 .05 .49 .487 .237 72.869 .001

Laissez-Faire
Leadership Identification -.58 .06 -51 .505 .255 80.606 .001
Internalization -.35 .05 -40 .399 .159 44.576 .001
Discussion
The first question addressed what the level of teachers' organizational commitment
was. Teachers' perceptions of organizational commitment were high in the internalization and
identification sub-dimensions but were low in the compliance sub-dimension.
School administrators' leadership styles affect teachers' well-being, which contributes to
their commitment to the school (Heidmets & Liik, 2014). In the study conducted in Estonian
public schools, it was found that school administrators 'higher transformational leadership
approach was closely related to the decrease in teachers' job quitting and burnout and thus the
coat of arms of their commitment to the school (Heidmets & Liik, 2014).
The second question addressed what kind of leadership styles that school administrators
showed according to teachers' perceptions. Findings showed that according to the teachers'
perceptions, the transformational leadership style was the most common, the transactional
leadership style was the second most common style, and the laissez-faire leadership style was
the third most common style.
In order to increase teachers' commitment to the school, the management should support
new teachers, especially when they are new to the profession. Supporting transformational
leadership approaches will contribute to teachers' holding on to their institutions in the
future (Kugiejko, 2015). According to the results of a meta-analytic study done by Kaya and
Selvitopu (2017), especially teacher commitment would be increased if managers possessed
transformational and instructional leadership styles. Considering that teachers who work with
school administrators with supportive leadership have higher organizational commitment

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
levels, school administrators 'more supportive leadership will increase teachers' commitment 215
to the school. (Çokluk & Yılmaz, 2010). Liu and Watson found that teachers' involvement in
decision-making mechanisms was one of the factors that increased their commitment to school.
Therefore, as transformational leaders, school leaders should consider teachers' involvement,
particularly in matters related to instructional and teacher development. (2020). Only in this
way can teachers' organizational commitment be strengthened and supported.
The third question of the research addressed whether there was a significant correlation
between school administrators' leadership styles and preschool teachers' organizational
commitment. Although there was a statistically significant negative correlation between
transformational leadership and compliance, the correlation between transformational leadership
and identification was a statistically significant positive one. Also, there was a statistically
significant positive correlation between transformational leadership and internalization. When
the relation between transactional leadership and three dimensions of teachers’ organizational
commitment sub-dimensions were analyzed, there was a statistically significant positive
correlation between transactional leadership and compliance. However, the correlation between
transactional leadership style and other two dimensions, identification, and internalization,
were statistically significant negative ones. The third leadership style, laissez-faire leadership,
had a statistically significant negative correlation with all sub-dimensions of organizational
commitment, compliance, identification, and internalization.
Akan and Yalçın (2015) found a statistically significant positive correlation between
the identification and internalization sub-dimensions of organizational commitment and
transformational leadership style. A statistically significant positive correlation was found
between teachers' perceptions of sustaining leadership style and compliance commitment and
a negative relation between identification commitment and internalization commitment. Yalçın
(2014) found a statistically significant positive correlation between transformational leadership
style and organizational commitment. Okçu (2011) found that transformational leadership had
a positive effect on organizational commitment.
Akan and Yalçın (2015) found a statistically significant negative correlation between
the transactional leadership style and the internalization and identification sub-dimensions of
organizational commitment. A statistically significant positive correlation was found between
teachers' perception of laissez-faire leadership style and compliance commitment and a
negative correlation between identification commitment and internalization commitment. Akan
and Yalçın (2015) found a statistically significant negative correlation between the laissez-faire
leadership style and the internalization and identification sub-dimensions of organizational
commitment. The correlation between administrators’ laissez-faire leadership style and
organizational commitment was high; therefore, the existing relation could affect the crucial
factors such as job satisfaction (Banjarnahor et al., 2018).
The last question of the research addressed whether the leadership styles of administrators
predicted the preschool teachers’ organizational commitment. The transformational leadership
style explains 22% of the total variance of the compliance dimension, 26% of the total variance
of the internalization dimension, and 40% of the identification dimension's total variance.
In comparison, the transactional leadership style explains 19% of the total variance of the
compliance dimension of organizational commitment, 15% of the total variance at the level
of identification dimension, and 7% of the total variance at the internalization dimension
level. The third type of leadership, the laissez-faire leadership style, explains 24% of the total
variance of the compliance dimension of organizational commitment, 26% of the total variance
of identification dimension, and 16% of the total variance of internalization dimension.
School administrators having transformational leadership characteristics contribute to
creating a positive attribution that teachers will develop a positive approach (Da’as, 2020).
Transformational leadership can increase the commitment levels of teachers, but instructional

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
216 leadership increases teachers' commitment to the school by increasing collaboration among
teachers (Cansoy, Parlar, & Polatcan, 2020).
Contrary to this result, it was found that the leadership styles of the administrators were
limited in the faculty's understanding and adoption of institutional goals. During organizational
change, by increasing the participation of lecturers in the decision-making process, lecturers
effectively adopt goals in organizational structures rather than managers' leadership styles
(Griffioen & de Jong, 2017). Although this research's transformational leadership has a
significant effect on teachers’ school commitment, there are also other types of leadership like
servant leadership that affects the organizational commitment levels of teachers (Cerit, 2010).
Another study determined that transformational leadership is related to teachers' organizational
trust but not to teacher commitment (Freeman & Fields, 2020). Like in this study, Khasawneh,
Omari, and Abu-Tineh found that transformational leadership had a strong effect on the
organizational commitment of vocational school teachers (2012). Leadership styles of school
principals affect both their organizational commitments and job satisfaction (Banjarnahor
et al., 2018). The findings of this study can be used to increase teacher commitment. If the
administrators of preschool schools want their teachers to be highly committed to the school, it
is recommended that they should adopt transactional leadership as their leadership style. When
different leadership styles are adopted, it would be helpful to consider the possible commitment
levels of teachers according to the results of the research.
Conclusions and Implications
It has been determined that preschool teachers show organizational commitment at the
highest level of internalization, medium level identification, and the lowest compliance level. It
was found that teachers perceived school administrators as transformational leaders, they were
indecisive in their perception as a supplementary leader, and their perception of liberator leaders
was weak. Pre-school teachers' perceptions of the transformational leadership style negatively
correlated with compliance commitment, and there was a significant positive relation between
identification commitment and internalization commitment.
This research was conducted by collecting data with scales and using different information
acquisition tools and methods. More detailed results can be achieved by using quantitative
and qualitative research studies together. Measures should be taken to ensure that educators
can work more effectively, using their transformational leadership qualities to manage people's
skills. School administrators can adopt participatory, transformational leadership understanding
in order to gain transformational leadership skills and to increase the organizational commitment
of teachers in the organization. The management of institutions can take in-service training
seminars to increase their awareness of the desired leadership behaviors.
In the education field, this research aimed to examine whether there was a relation
between administrators' leadership styles and preschool teachers' organizational commitment
levels according to preschool teachers' perceptions. Because in preschool institutions,
teachers' institutional commitment is essential for the development of students. It is vital for
administrators to be aware of this and to know the impact of the leadership styles they adopt on
teachers' commitments.
This research has different limitations. The first of these limitations is that the participants
are only teachers working in public schools. Besides, the fact that the study is local and that it
is done in a relatively short time is among the limitations of the study.
Despite the limitations mentioned above, it presents essential findings and comments
for policymakers in the selection and training of school administrators, preschool teachers,
and educational administrators in order to reveal the correlation between preschool teachers’
organizational commitment levels and school administrators' perceived leadership styles. In

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
order to help teachers and administrators to understand each other in preschool institutions, it 217
appears that the leadership styles of administrators affect them in terms of teachers.
Acknowledgments
The authors would like to thank all teachers who participated in the research and Carol
Williams; Kasım Varlı for the English language review. The second author's graduate thesis was
used to write this article, which the first author supervised.
References
Abasilim, U., Gberevbie, D., & Osibanjo, O. (2019). Leadership styles and
employees’ commitment: Empirical evidence from Nigeria. SAGE Open,
9 ( 3 ) , 1 – 1 5 . h t t p s : / / d o i . o rg / 1 0 . 11 7 7 % 2 F 2 1 5 8 2 4 4 0 1 9 8 6 6 2 8 7
Akan, D., Yalçın, S., & Yıldırım, İ. (2017, May 11-13). Okul yöneticilerinin liderlik stilleri ile
öğretmenlerin pozitif psikolojik sermaye düzeyleri arasındaki ilişkinin incelenmesi. [Investigation
of the relationship between leadership styles of school administrators and teachers' positive
psychological capital levels]. 12th International Educational Management Symposium, Ankara,
Turkey.
Akan, D., Yıldırım, İ., & Yalçın, S. (2014). Okul Müdürlerinin Liderlik Stili Ölçeğinin Geliştirilmesi
(OMLSÖ) [Developing leadership style scale of school principals (OMLSÖ)]. Elektronik Sosyal
Bilimler Journal, 13(51), 392-415. https://doi.org/10.17755/esosder.28743
Akan, D., & Yalçın, S. (2015). Okul yöneticilerinin liderlik stilleri ile öğretmenlerin örgütsel bağlılıkları
arasındaki ilişkinin incelenmesi [Examining the relationship between school administrators
'leadership styles and teachers' organizational commitment]. Journal of Education and Humanities:
Theory and Practice, 6(11), 123-150.
Allen, N. J., & Meyer, J. P. (1990). The measurement and antecedents of affective, continuance, and
normative commitment to the organization. Journal of Occupational Psychology, 63(1),
1-18. https://doi.org/10.1111/j.2044-8325.1990.tb00506.x
Avcı, A. (2015). Örgütsel vatandaşlık davranışları: kavramsal gelışımı ve eğıtım örgütlerı açısından
etkıleri [Organizational citizenship behaviors: Its conceptual development and its effects on
educational organizations]. Hasan Ali Yücel Eğitim Fakültesi Journal, 12(2), 11-26.
Avcı, A. (2017). Örgütsel bağlılık: Kavramsal çerçevesi ve eğitim örgütleri açısından etkileri
[Organizational commitment: Its conceptual framework and its effects on educational
organizations]. International Periodical for the Languages, Literature and History of Turkish or
Turkic, 12(6), 55-76. http://dx.doi.org/10.7827/TurkishStudies.11535
Balay, R. (2000). Özel ve resmi liselerde yönetici ve öğretmenlerin örgütsel bağlılığı [Organizational
commitment of administrators and teachers in private and public high schools] [Unpublished
doctoral dissertation]. Ankara University.
Balcı, A. (2003). Örgütsel sosyalleşme kuram strateji ve taktikler [Organizational socialization theory,
strategy and tactics]. Pegem A Publications.
Banjarnahor, H., Hutabarat, W., Sibuea, A. M., & Situmorang, M. (2018). Job satisfaction as a mediator
between directive and participatory leadership styles toward organizational commitment.
International Journal of Instruction, 11(4), 869-888. https://doi.org/10.12973/iji.2018.11455a
Celep, C. (2014). Eğitim örgütlerinde örgütsel adanma [Organizational commitment in educational
organizations]. Nobel Publications.
Cerit, Y. (2010). The effects of servant leadership on teachers’ organizational commitment in
primary schools in Turkey. International Journal of Leadership in Education, 13(3), 301-
317. https://doi.org/10.1080/13603124.2010.496933
Chiun, Lo, M., Ramayah, T., & Wei Min, H. (2009). Leadership styles and organizational commitment: A
test on Malaysia manufacturing industry. African Journal of Marketing Management, 1(6), 133-
139. https://doi.org/10.1080/13602380903355676
Çetin, M. (2004). Örgüt kültürü ve örgütsel bağlılık [Organizational commitment in educational
organizations]. Nobel Publications.

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
218 Çokluk, Ö., & Yılmaz, K. (2010). The relationship between leadership behavior and organizational
commitment in Turkish primary schools. Bilig, 54, 75-92.
Da’as, R. (2020). Between principals’ and a teacher’s perspective taking: the role of transformational
and transactional leadership styles. International Journal of Leadership in Education, 1-23.
https://doi.org/10.1080/13603124.2020.1850871
Erceylan, N. (2010). Yöneticilerin liderlik davranışlarının çalışanların örgütsel bağlılıkları üzerindeki
etkileri ve bir araştırma [The effects of managers 'leadership behaviors on employees'
organizational commitment and a research] [Unpublished master’s thesis]. İnönü University.
Eren, E. (2018). Stratejik yönetim ve işletme politikası [Strategic management and business policy]. Beta
Publications.
Freeman, G. T., & Fields, D. (2020). School leadership in an urban context: Complicating notions of effective
principal leadership, organizational setting, and teacher commitment to students. International
Journal of Leadership in Education, 1-21. https://doi.org/10.1080/13603124.2020.1818133
Gardner, A. (2018). The relationship between organizational commitment and transformational and
transactional leadership styles in government contract employees [Unpublished doctoral
dissertation]. Liberty University.
George, D., & Mallery, M. (2010). SPSS for windows step by step: A Simple Guide and Reference, 17.0
Update, (10th Edition). Pearson.
Griffioen, D.M.E., &. de Jong, U. (2017) The influence of direct executive managers on lecturers’
perceptions on new organizational aims in times of academic drift. International Journal of
Leadership in Education, 20(4), 451-467. https://doi.org/10.1080/13603124.2015.1066867
Heidmets, M., & Liik, K. (2014). School principals’ leadership style and teachers’ subjective well-being
at school. Problems of Education in the 21st Century, 62, 40-50.
Hoşcan, Y., Oktal, Ö., Hepkul, A., Kağnıcıoğlu, H., & Sevim, A. (2004). Yönetim bilgi sistemi
[Management information system]. AÖF Publications.
İbicioğlu, H. (2000). Örgütsel bağlılıkta paradigmatik uyumun yeri [The place of paradigmatic fit in
organizational commitment]. Dokuz Eylül Üniversitesi İktisadi İdari Bilimler Fakültesi Journal,
15(1), 13-22.
İmamoğlu, G. (2011). İlköğretim okulu öğretmenlerinin örgütsel bağlılık düzeyleri ve örgütsel adalet
algıları arasındaki ilişki [The relationship between primary school teachers' organizational
commitment levels and their perceptions of organizational justice] [Unpublished master’s thesis].
Gazi University.
Kantos, E. Z. (2011). Örgüt metaforlarında liderlik: Kavramsal bir çözümleme [Leadership in
organizational metaphors: A conceptual analysis]. Eğitim Bilimleri Araştırmaları Journal, 1(1),
135-158.
Karasar, N. (2017). Bilimsel araştırma yöntemi [Leadership in organizational metaphors: a conceptual
analysis]. Nobel Publications.
Karip, E. (1998). Dönüşümcü liderlik [Transformational leadership]. Kuram ve Uygulamada Eğitim
Yönetimi Journal, 4(16). https://dergipark.org.tr/en/pub/kuey/issue/10380/127019
Kaya, M., & Selvitop, A. (2017). Leadership and organizational commitment in educational context: A
meta- analytic review. Dicle University Journal of Ziya Gökalp Faculty of Education, (31), 719-
728. https://doi.org/10.14582/DUZGEF.1822
Khasawneh, S., Omari, A., & Abu-Tineh, A. M. (2012). The relationship between
transformational leadership and organizational commitment: The case for vocational
teachers in Jordan. Educational Management Administration & Leadership, 40(4),
494–508. https://doi.org/10.1177/1741143212438217
Kugiejko, M. (2015). A professional skill set of a teacher and university lecturer in relation to the
causes and prevention of job burnout. Problems of Education in the 21st Century, 67, 40-51.
http://www.scientiasocialis.lt/pec/files/pdf/vol67/40-51.Kugiejko_Vol.67_PEC.pdf
Liu, Y., & Watson, S. (2020). Whose leadership role is more substantial for teacher professional
collaboration, job satisfaction and organizational commitment: A lens of distributed leadership.
International Journal of Leadership in Education, 1-29. https://doi.org/10.1080/13603124.2020
.1820580
Okçu, V. (2011). Okul yöneticilerinin liderlik stilleri ile öğretmenlerin örgütsel bağlılıkları ve yıldırtma
yaşama düzeyleri arasındaki ilişkilerin incelenmesi [Examining the relationship between
leadership styles of school administrators and teachers' organizational commitment and
intimidation] [Unpublished master’s thesis]. Gazi University.

styles
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
O'Reilly, C. A., & Chatman, J. (1986). Organizational commitment and psychological attachment: The 219
effects of compliance, identification, and internalization on prosocial behavior. Journal of Applied
Psychology, 71(3), 492–499. https://doi.org/10.1037/0021-9010.71.3.492
Özgener, S., and Kılıç, G. (2009, May 21- 23). Eğitim kurumlarında liderlik tarzları ve örgüt kültürünün
performans üzerindeki etkisi [Leadership styles in educational institutions and the impact of
organizational culture on performance]. 17th National Management and Organization Congress,
Eskişehir, Turkey. http://w3.balikesir.edu.tr/~seymen/yonetim.pdf
Raosoft. (2018). Raosoft sample size calculator. [Software]. http://www.raosoft.com/samplesize.html
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th Edition). Allyn and Bacon.
Türkmen, F. (2016). Ortaokul yöneticilerinin hizmetkar liderlik davranışlarının öğretmenlerin örgütsel
bağlılıkları üzerindeki etkisi [The effect of middle school administrators 'servant leadership
behaviors on teachers' organizational commitment] [Unpublished master’s thesis]. Ondokuz
Mayıs University.
Yalçın, S. (2014). Öğretmenlerin algılarına göre okul yöneticilerinin liderlik stilleri ile öğretmenlerin
iş yaşam kalitesi ve örgütsel bağlılık düzeyleri arasındaki ilişkinin incelenmesi [Investigation of
the relationship between school administrators' leadership styles and teachers' work quality of
life and organizational commitment according to teachers' perceptions] [Unpublished doctoral
dissertation]. Ataturk University.
Yıldırım, C. (2006). Okul müdürlerinin liderlik stillerinin örgütsel sağlık üzerindeki 211 etkisi [211
effects of school principals' leadership styles on organizational health]. [Unpublished master’s
thesis]. Gazi University.
Zangora, G. (2001). Organizational commitment: A concept analysis. Nursing Forum, 36(2), 14–21.
https://doi.org/10.1111/j.1744-6198.2001.tb01179.x
Received: December 26, 2020 Accepted: March 24, 2021
Cite as: Kırkıç, K. A., & Balci, T. D. (2021). Organizational commitment levels of preschool
teachers and administrators' leadership styles. Problems of Education in the 21st Century,
79(2), 207-219. https://doi.org/10.33225/pec/21.79.207
Kamil Arif Kırkıç PhD, Assistant Professor, Istanbul Sabahattin Zaim University, Halkalı
(Corresponding author) Cad. No: 281 Halkalı Küçükçekmece, Istanbul, Turkey.
E-mail: kamil.kirkic@izu.edu.tr
ORCID: https://orcid.org/0000-0002-8902-437X
Türkan Demir Balcı Ministry of National Education, Turkey.

E-mail: turkkan_28@hotmail.com

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
220
THE EFFECT OF EXPERIENCE ON
REDUCING THE OVERCONFIDENCE
EFFECT IN TEACHERS
Robert Krause
Constantine the Philosopher University in Nitra, Slovakia
E-mail: rkrause@ukf.sk
Abstract
Being overconfident in one’s abilities is connected to poor self-judgment and it is essential to find ways
how to reduce this phenomenon. The aim of the study was to research whether the act of taking a reading
literacy test can be an effective means of reducing the overconfidence effect in the sample of Slovak
teachers. Using purposive sampling method, a sample of 225 teachers from across school levels was
obtained. The levels of overestimation (OE) and overplacement (OPE) effects were compared before
and after having undergone the reading literacy test. The results showed that even just taking the test
can reduce OE and to a lesser degree also OPE in all participants regardless of the feedback, which was
in contrast to the original belief that just the experience itself does not affect overconfidence. Greater
reduction in OE was observed in less competent participants, whereas none was observed among the
more competent. These results showed that experience is also a form of debiasing.
Keywords: overconfidence effect, overestimation effect, effect of experience, debiasing overconfidence
Introduction
Teachers have the task of passing on the knowledge to their students, but they can be
overconfident and pass it on wrongly. The role of the teacher is very important for the development
of the student, as his teaching activity influences the student's personality. However, if teachers
have a non-objective self-assessment, there is an increased likelihood that the teacher will not
be able to motivate his student to objective self-reflection. Many studies which have taken place
over the last 40 years, have shown that people are overconfident (Boyd, 2014; Bracha & Brown,
2010; Ghose, 2013; Gilovich et al., 2002; Kahneman & Tversky, 1979; Kahneman & Tversky,
2004; Maratolli & Richardson, 1988; Tversky & Kahneman, 1974; Whitlestone, 2012). Many
researchers such as Whitlestone (2012), Maratolli and Richardson (1998), Lichtenstein and
Fischhoff (1977), Svenson (1981) and McGraw et al. (2004) have pointed out that not only
laymen are overconfident, but experts too. Some research studies (Lichtenstein et al., 1982;
Russo & Schoemaker, 1992) have even shown that experts often think of themselves as better
than they really are and at the same time as better than their counterparts.
Kearney and Sheffer (2015) have illustrated that college teachers are often convinced that
their lectures are more comprehensible (overestimation effect) and that the performance they
have given is better than that given by their colleagues (overplacement effect). Cross (1977) has
documented superiority (overplacement effect) in college teachers as early as the 1980s, when
he found that teachers thought they were better than their colleagues.
Kruger and Dunning (1999) have pointed out that competent individuals tend to
underestimate their abilities and performance, and incompetent individuals overestimate those.
It seems that paying attention to the overconfidence effect is also important in professionals.
This study reflects the view of Olsson (2014) that the overconfidence effect is formed
by three constructs, and these represent the ways in which overconfidence effect is studied.

Robert KRAUSE. The effect of experience on reducing the overconfidence effect in teachers
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
First construct is overplacement effect, which is about individuals overestimating their abilities 221
and performance when compared to other people (Barbara & Odean, 2001; McGraw et al.,
2004; Shiller, 2005). Second component is overestimation effect, which pertains to individuals
thinking that their abilities are better than they really are (Arkes et. al., 1981; Deaves et.al., 2008;
Oskamp, 1965). The last construct is calibration of probabilities, which is about individuals
overestimating their predictions (Hoorens, 1995; Svenson, 1981).
In addition, this study considers the research of Kruger and Dunning (1999) pointing
to the role of competence in relation to the overestimation and underestimation of abilities.
McKenzie (1997) found that a decrease in overconfidence also occurs when people are forced
to consider alternative outcomes. Koriat et.al (1980) has shown that excessive self-confidence
can be reduced even when participants are asked to compile a list of counterarguments to their
estimates before writing their final estimate. Kruger and Dunning (1999) have pointed out
that experience is not a debiasing method, whereas Krueger and Mueller (2002) have argued
that experience can also affect one's self-evaluation if the individual is prepared for it. So, the
question remains whether it is possible to reduce the overconfidence effect only by experience
alone?
Research Problem
Overconfidence is connected to poor self-judgment and finding ways of affecting it are

of importance among teaching professionals. It was originally believed that experience does not
calibrate the perception of oneself and one’s performance (Kruger & Dunning, 1999), but later
it was found that feedback can influence self-evaluation of participants, if it is given well (Krajč
& Ortmann, 2008). This study aimed to answer the question whether experience with the task
can be an effective means of reducing overconfidence in teachers as it is also a sort of feedback,
and can serve as a form of debiasing. Specifically, how does the experience with Reading
Literacy Test (RLT) affect the overestimation and overplacement effects among teachers?
Research Focus
Stanovich (2011) was convinced that debiasing can occur even if an individual is able
to suppress his primary automatic responses. At the same time, Stanovich (2011) added that
in order for debiasing to take place, it is necessary for the individual to have space in the
cognitive capacity that would allow him to better detect distortion and subsequently find a
solution. Larrick (1993) argued that we can look at debiasing in two ways. The first of them is
characterized by the fact that it focuses on the decision-maker himself, so that the level of his
knowledge base increases, for instance through education, or by providing him with tools to
streamline his decision-making process. The second approach is different in that it focuses on
the environment in which the individual makes decisions by creating optimal conditions in the
environment that positively affect the decision-making process. This research aims to verify
the notion of Larrick (1993) whether experience with RLT does increase the level of learning
in the individual, and the notion of Kruger and Dunning (1999) whether it would correct the
underestimating or overestimating tendencies.
The aim of the research was to study whether the experience with RLT affects
overestimation and overplacement effects, and if the degree of competence plays a role in this
process. The background of the study was research by Kruger and Dunning (1999), who pointed
out that if a competent individual receives feedback, his perception becomes closer to reality,

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
222 but if an incompetent individual receives objective feedback, his perception does not change.
Ramaprasad (1983) argued that in order for a person to know a blind spot in his perception,
it is important to receive feedback that would inform him of the very difference between the
expected and actual level of his abilities or performance. Nordquist (2014) added that even
through feedback, an individual can gain knowledge that his abilities and competencies may
not really be what he considers them to be. Is an experience also a form of feedback? If so, how
does it affect the less competent and how more competent?
Following research questions were formulated:
RQ1: Does the experience with RLT affect OE?

RQ2: Does the experience with RLT affect OPE?
RQ3: Does the effect of the RLT experience on the OE depend on the level of competence?
RQ4: Does the effect of the RLT experience on the OPE depend on the level of competence?
General Background
This study was part of the larger research project APVV-0361-12 titled “Rozhodovanie
profesionálov: Procesuálne, osobnostné a sociálne aspekty [Decision making of professionals:
Processual, personality, and social aspects]” taking place at the Slovak Academy of Sciences,
which focused on different ways of reducing the overconfidence effect. To be able to answer the
research question about causal relationship, experimental methodology with within-subjects
research design was chosen, and the study was conducted using online platform Qualtrics.com.
The participants were asked to estimate their skills and performance prior and after having
completed the RLT.
Sample
Purposive sampling method was used, and 14 Slovak schools were contacted to participate
in the study. Those schools were found in the database of National Institute for Certified
Educational Measurement and were contacted based on the previous research cooperation.
The goal was to recruit as many participants as possible to achieve sufficient statistical power.
Eventually, total of nine schools agreed to partake in the research and the final sample consisted
of 223 teachers, which is the best possible result as research participants are notoriously hard
to find. The mean age of teachers was 45.48 (SD = 10.13) and the age range was 24-69 years.
Teachers came from the variety of school levels: primary (n = 38), lower-secondary (n = 82),
upper-secondary (n = 95), and universities (n = 8). Participation of the research was voluntary,
and teachers filled out the test battery in the presence of the researcher at school, while some
individual participants partook the experiment from home.
Reading Literacy Test (RLT)
The test was created by Mgr. Branislav Hudcovský from the National Institute for Certified
Educational Measurements (NÚCEM) in Slovakia for the purpose of this research based on
the pilot testing on the population of high school seniors, which NÚCEM regularly organizes
(Krause & Kurincová Čavojová, 2018). The test consists of three stories which participants

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
ought to read and then answer 10 questions pertaining to the texts (30 questions total). Only first 223
of the three stories was used to test for the effect of experience. The psychometric properties of
the test are not known; however, those tests are used in regular testing of high school students.
For the purpose of this research, it is, therefore, less important whether it actually measures
what it is supposed to measure, because it is used in practical settings, nonetheless.
Overestimation effect (OE)
Overestimation effect was measured by asking participants to estimate the number of

correct answers on the RLT on a scale from 0 to 10.
Overplacement effect (OPE)
Overplacement was measured by asking participants to estimate their placement among

other participants on a scale from 1 to 100 where 1 is the best and 100 is the worst placement.
Research procedure
Participants first answered demographic questions about their age, gender and at what
school level they were teaching and continued to the questions on OE and OPE, which were
used based on the previous research in the field that uses two questions to measure these two
effects (Olsson 2014). Afterwards, they moved on to completing the Reading Literacy Test
and questions related to what the participants read. Finally, after reading and then answering
all the questions, the participants again filled in the questions on OE and OPE. Based on the
recommendations of Olsson (2014), OE was measured as comparison between expected
performance and actual one, and OPE as self-evaluation of one’s performance relative to other
people.
Data Analysis
The statistical methods used to analyse data are listed in Table 1. The data were analysed
using Statistical Package for the Social Sciences (SPSS), version 20.
Table 1
Methods of Answering the Research Questions

Related t-test with OE before and after RLT as dependent
RQ1: Does the experience with RLT affect OE? variables. Cohen’s d was computed to compare the effect
sizes of RLT on reducing OE and OPE.
Related t-test with OPE before and after RLT as dependent

RQ2: Does the experience with RLT affect OPE? variables. Cohen’s d was computed to compare the effect
sizes of RLT on reducing OE and OPE.
RQ3: Does the impact of the RLT experience on the Related t-tests with OE and OPE before and after RLT
OE depend on the level of competence? as dependent variables in groups of participants with
RQ4: Does the impact of the RLT experience on the different levels of competence with cut-offs on 33rd and 66th
OPE depend on the level of competence? percentiles.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
224 Research Results
Effect of Experience with RLT on OE and OPE
To check whether the actual experience with the Reading Literacy Test will serve as a form
of feedback and will affect overplacement and overestimation (RQ1,RQ2), the overplacement
and overestimation indices before and after taking RLT were compared using t-test for related
scores. The results showed that taking RLT significantly reduced overestimation (M1 = 1.21,
SD1 = 2.31 vs. M2 = 0.13, SD2 = 2.02), t (222) = 8.41, p <.001, d = .80. The results also show
that the experience with the Reading Literacy Test decreased overplacement (M1 = 5.32, SD1
= 33.03 vs M2 = 1.92, SD2 = 33.32), t (222) = 2.45, p = .015, d = .23), which was a small
difference. Comparison of effect sizes indicates that experience affects overestimation to the
greater degree than overplacement.
Effect of Experience with RLT on OE and OPE in Relation to Competence
To check whether differences in pre- and post-test OE and OPE were differently
pronounced based on the competence levels (RQ3, RQ4), individual related t-tests were run in
the groups of participants with different levels of competence based on the cut-offs on 33rd and
66th percentiles.
Experience with RLT in less competent participants significantly reduced OE (M1 = 2.36,
SD1 = 1.85 vs M2 = 0.79, SD2 = 1.92), t (142) = 10.67, p <.001, d = 1.27, which was a big effect.
The results also indicated that the experience with RLT in less competent participants did not
affect OPE (p >.05). At the same time, the analysis of more competent participants showed that
the experience with RLT did not affect OE or OPE (p >.05). More competent, regardless of their
experience with RLT, tended to underestimate themselves.
Comparison Differences in OE and OPE among Differently Competent Individuals
As part of the analyses, less and more competent participants were compared in the OE
and OPE using independent-samples t-test. In comparing all participants, it was found that
less competent participants (M = 2.36, SD = 1.85) were significantly more overestimated (OE)
than more competent (M = -0.84, SD = 1.50), t (221) = 14.02, p <.001, d = 1.33. Similarly, less
competent participants (M = 20.32, SD = 27.3) showed significantly greater overplacement
effect in the first measurement than the more competent (M = -21.48, SD = 24.28), t (221) =
11.40, p <.001, d = 1.08.
Discussion
The aim of the study was to see whether the experience with RLT would affect the
overestimation and overplacement effects and to examine whether the degree of competence
played a role in this process. Results show that experience with RLT influenced OE and OPE
in the whole research sample with a very large effect size. Competence played a significant
role with less competent participants being overestimated before completing the RLT and more
competent participants underestimated. However, after the experience with the test, competence
did not play a role in OE, as experience with RLT inhibited OE in all participants, with the
difference that in less competent it led to a more realistic estimate (although still overestimated)
and in more competent to even more underestimating.
In the experience with RLT among the more competent people, it was found that more
competent participants underestimated themselves even more after the experience with RLT

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
than before completing the RLT. This can potentially be explained by Hilton et al. (2011) 225
who pointed that the complexity of the presented tasks also played an important role in the
overconfidence effect. They wrote that for complex tasks individuals rated their estimates as
below average when compared to other people. And RLT as such can be viewed as a complex
task.
Bazerman (2002) showed that providing individual with information about his cognitive
biases can be a sufficient form of debiasing. In this study, the participants were able to gain
not only a cognitive insight, but also own emotional experience from the RLT, which is an
important part in the reduction of the overconfidence effect (Bazerman, 2002). The results of
this study are also aligned with Stanovich (2011), who claimed that debiasing can occur when
an individual is able to suppress his primary automatic responses, and by the use of conscious
processes find mental cues which lead to the elimination of the present cognitive distortion.
Ramaprasad (1983) writes that in order for the individual to be aware of a blind spot in one’s
self-perception, it is important that he be given feedback about the discrepancies between the
expected and actual performance. The present study had experience with RLT as such feedback
as participants could compare the expected performance with the actual one.
Pintrich and Schunk (2002) discuss that individuals who are actively involved in
receiving feedback show greater self-efficacy afterwards, which motivates them to perform
better in the future. In the case of this study, participants were active agents of feedback, as
they were not given it from the environment, but based on their own experience with RLT. The
study by Wiliam (2011) showed that concrete feedback leads to great improvement in the future
testing, as compared to the feedback given on a point-scale. In this study, the feedback was also
concrete, which the participants addressed to themselves based on their own experience.
It is worth considering the findings of the meta-analysis by Smither et al. (2005) who
discovered that in order for the feedback to be the most effective, it is essential that the message
suggests that change is necessary, but also that the individual is open to change and perceives
an inner desire to change his behaviour while believing that change is possible. Feedback can be
of different valences, which play a role in the adjustment of thoughts and behaviour (Jaworski
& Kohli, 1991). It was the valence of the feedback, which was the core of by Krajč and Ortman
(2008) criticism of the original research by Kruger and Dunning (1999) by showing that the
incompetent do not adjust their estimates because of the metacognitive deficit, as was believed
by Kruger and Dunning (1999), but because the feedback they were given was negative for
them.
Kruger and Dunning (1999) originally wrote that if individuals are confronted with
objective results, they do not adjust their abilities and performance. This study shows that such
statement is only partially true. The results of this study show that even the very experience that
participants have allows them to see themselves better in the context of their performance, even
if they are among the less competent group. However, ignorance of the performance of others
makes them, despite their initial estimate, still think they are better off than other people. In
other words, it is not so much about their metacognitive deficit as Kruger and Dunning (1999)
believe, which prevents them from seeing their level of incompetence, but rather ignorance of
other people's performance.
However, this may also explain what Burson (2006), Krueger and Mueller (2002) and
Yarkoni (2010) argued that even if incompetent individuals are overestimated, they do not rank
higher than competent ones. In other words, an incompetent person does not think he is smarter
than a more competent one. As part of future research, less competent participants should be
confronted with other people's objective results to discern whether they would continue to
believe that they are better off than they actually are, despite having an objective knowledge of
other people's performance.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
226 Conclusions and Implications
The present study supports the original Kruger and Dunning’s findings by showing
that less competent participants believe they are better off than others and have the tendency
to overestimate their performance (OPE), while the more competence have the tendency
to underestimate. On the other hand, this study disproves their claim that it is due to the
metacognitive deficit, which prevents individuals in adjusting their self-perception, as this study
showed that even own experience can lead to the change in perception of one’s performance
(OE) in the less competent participants.
Based on the results of this study, the criticism of original Kruger and Dunning’s study
on the basis of not considering the statistical explanation of the wrong self-evaluation and
not researching the differences in metacognition between more and less competent, seems
justified. The inaccuracy in perception is not always needed to be explained by the deficit in
metacognition, but also as combination of universal cognitive distortions, such as “better than
average” or “regression to the mean.”
The study showed that own test experience can also be type of debiasing. It seems
that just taking the RLT affected the overconfidence, as the participants largely overestimated
themselves before the test, but after the experience with the test their overestimation decreased.
It was found that the experience with RLT in less competent participants significantly reduced
overestimation effect. The results of this study also show that one's own experience can change
perception of oneself and one’s abilities, even if one is less competent when compared to other
people. The results of this study can be summarized in such a way that even having an experience
can change an individual's perception, even if the individual is less competent. Therefore, it is
important to give people the opportunity to adjust their own perceptions, for example in the
form of experience, as it was shown to also be a form of debiasing.
Regarding teachers, one way to provide them with experience is to let them take the tests
they are teaching for. This study used a Reading Literacy Test designed for high school seniors
and showed that teachers can be overconfident in their ability on this test. It is suggested that
teaching professionals are given the opportunity to calibrate their believed skills with their
actual performance, as this can reduce their overconfidence.
References
Bazerman, M. (2002). Judgment in managerial decision making. Wiley & Sons.

Boyd, S. (2014, August 14). 93% of us think we are above average. Gigaom Research. https://gigaom.
com/2014/08/19/93-of-us-think-we-are-above-average/
Bracha, A., & Brown, D. J. (2010). Affective decision making: A theory of optimism bias. Games and
Economic Behavior, 75(1), 67-80. https://doi.org/10.1016/j.geb.2011.11.004
Burson, K., Larrick, R., & Klayman, J. (2006). Skilled or unskilled, but still unaware of it: How
perceptions of difficulty drive miscalibration in relative comparisons. Journal of Personality and
Social Psychology, 90(1), 60-77. https://doi.org/10.1037/0022-3514.90.1.60
Cross, K. P. (1977). Not can, but will college teaching be improved? New Directions for Higher Education.
17, 1-15. https://doi.org/10.1002/he.36919771703
Fischhoff, B. (1982). Debiasing. In D. Kahneman, P. Slovic, A. Tversky (Eds.), Judgment under
uncertainty: Heuristics and biases (pp. 422-444). Cambridge University Press.
Ghose, T. (2013, February 06). Why we're all above average. Live Science. http://.livescience.com/26914-
why-we-are-all-above-average.html
Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of intuitive
judgment. Cambridge University Press.
Hilton, D., Regner, I., Cabantous, L., Charalambides, L., & Vautier, S. (2011). Do positive illusions
predict overconfidence in judgment? A test usinginterval production and probability
evaluation measures of miscalibration. Journal of Behavioral Decision Making, 24(2), 117-
139. https://doi.org/10.1002/bdm.678

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Jaworski, B. J., & Kohli, A. K. (1991). Supervisory feedback: Alternative types and their impact 227
on salespeople's performance and satisfaction. Journal of Marketing, 28(2), 190-201.
https://doi.org/10.2307/3172807
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica,
47(2), 263-291. https://doi.org/10.2307/1914185
Kahneman, D., & Tversky, A. (2004). Conflict resolution: A cognitive perspective. In E. Shafir (Ed.),
Preference, belief, and similarity: Selected writings/by Amos Tversky (pp. 729-746). Massachusetts
Institute of Technology.
K o r i a t , A . , L i c h t e n s t e i n , S . , F i s c h h o ff , B . ( 1 9 8 0 ) . R e a s o n s f o r c o n f i d e n c e . J o u r n a l
of Experimental Psychology: Human Learning and Memory, 6(2), 107-
11 8 . h t t p s : / / d o i . o rg / 1 0 . 1 0 3 7 / 0 2 7 8 - 7 3 9 3 . 6 . 2 . 1 0 7
Krajč, M., & Ortmann, A. (2008). Are the unskilled really that unaware? An alternative explanation.
Journal of Economic Psychology, 29(5), 724-738. https://doi.org/10.1016/j.joep.2007.12.006
Krause, R., & Kurincová Čavojová, V. (2018). Vzťah efektu nadmernej sebadôvery a osobnostných
premenných [Relationship of overconfidence effect and personality variables]. In Č. Sašinka, A.
Strnadová, Z. Šmideková, V. Juřík (Eds.), Kognice a umělý život (pp. 40-41). Flow, z.s.
Krueger, J., & Mueller, R. A. (2002). Unskilled, unaware, or both? The contribution of social-perceptual
skills and statistical regression to self-enhancement biases. Journal of Personality and Social
Psychology, 82(2), 180-188. https://doi.org/10.1037//0022-3514.82.2.180-188
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own
incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology,
77(6), 1121-1134. https://doi.org/10.1037//0022-3514.77.6.1121
Larrick, R. P. (1993). Motivational factors in decision theories: The role of self-protection. Psychological
Bulletin, 113(3), 440–450. https://doi.org/10.1037/0033-2909.113.3.440
Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much
they know? The calibration of probability judgments. Organizational Behavior and Human
Performance, 20(2), 159-183. https://doi.org/10.1016/0030-5073(77)90001-0
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of subjective probabilities: The state
of the art up to 1980. In D. Kahneman, P. Slovic, A. Tversky (Eds.), Judgment under uncertainty:
Heuristics and biases (306-334). Cambridge University Press.
Marottoli , R. A., & Richardson, E. D. (1988). Confidence in, and self-rating of, driving ability among
older drivers. Accident Analysis and Prevention, 30(3), 331–336. https://doi.org/10.1016/S0001-
4575(97)00100-0
Mcgraw, P., Mellers, B. A., & Ritov, I. (2004). The affective costs of overconfidence. Journal of Behavioral
Decision Making, 17(4), 281-295. https://doi.org/10.1002/bdm.472
Olsson, H. (2014). Measuring overconfidence: Methodological problems and statistical artifacts. Journal
of Business Research, 67(8), 1766-1770. https://doi.org/10.1016/j.jbusres.2014.03.002
Oskamp, S. (1965). Overconfidence in case-study judgments. Journal of Clinical Psychology, 29(3), 261-
265. https://doi.org/10.1037/h0022125
Pintrich, P. R., & Schunk, D. H. (2002). Motivation in education: Theory, research, and applications (2nd
ed.). Prentice Hall.
Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science, 28(1), 4-13.
https://doi.org/10.1002/bs.3830280103
Russo, J. E., & Schoemaker, P. J. H. (1992). Managing overconfidence. Sloan Management Review,
33(2), 7-17.
Smither, J. W., London, M., & Reilly, R. (2005). Does performance improve following multisource
feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel
Psychology, 58(1), 33-66. https://doi.org/10.1111/j.1744-6570.2005.514_1.x
Stanovich, K. E. (2011). Rationality and the reflective mind. Oxford University Press.
Svenson, O. (1981). Are we all less risky and more skillful than our fellow drivers? Acta Psychologica,
47, 143-148. https://doi.org/10.1016/0001-6918(81)90005-6
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science,
185(4157), 1124-1131. https://doi.org/10.1126/science.185.4157.1124
Yarkoni, T. (2010, July 7). What the Dunning-Kruger effect is and isn’t. Talyarkoni.org. http://
www.talyarkoni.org/blog/2010/07/07/what-the-dunning-kruger-effect-is-and-isnt/

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
228 Whittlestone, J. (2012, November 27). Do you think you’re better than average? 80,000 Hours.
https://80000hours.org/2012/11/do-you-think-you-re-better-than-average/
Wiliam, D. (2011). Embedded formative assessment. Solution Tree Press.
Received: January 19, 2020 Accepted: April 02, 2021
Cite as: Krause, R. (2021). The effect of experience on reducing the

overconfidence effect in teachers. Problems of Education in the 21 st Century,
79(2), 220-228. https://doi.org/10.33225/pec/21.79.220
Robert Krause PhD, Research Assistant, Department of Educational and School Psychology,
Constantine The Philosopher University in Nitra, Tr. A. Hlinku 1, 94901 Nitra,
Slovakia.
E-mail: rkrause@ukf.sk
Website: https://www.ukf.sk/en/

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
STUDENTS’ PERCEPTIONS ON DISTANCE 229
EDUCATION: A CASE STUDY IN

MOZAMBIQUE
Alzira Manuel, Domingos Buque, Rosário Quive
Eduardo Mondlane University, Mozambique
E-mail: alziramanuel92@gmail.com, domingos.buque@gmail.com,
rosquive@gmail.com
Abstract
Education in all societies is one of the requisites for changes and development, particularly in the current
context influenced by globalization and rapid changes in which knowledge and technology became
essential tools for life. In this context, Mozambique has been and continues promoting education, including
Classroom Teaching Adult Education and Distance Adult Education (DAE) to satisfy people’s learning
needs and improve their capacities towards changes. DAE is considered a way to open opportunities of
education for youth and adults who by different reasons cannot participate in classroom teaching. This
study seeks to explore the perceptions of youth and adult students in DAE at Eduardo Mondlane University
(EMU), about their learning process. The study applied qualitative approach, based on semi-structured
interview administrated to twelve students from the 3rd and 4th years of the undergraduate program in
Organization and Management of Education in Distance Education (DE) modality. The findings of this
study point out that DAE is perceived as a relevant and useful modality of education, particularly for adult
learners. However, the study identifies contextual challenges associated to individual and professional
issues, pedagogical and organizational issues as well as gender division of labor within the families,
impacting the student’s performance in DE in the selected higher education institution.
Keywords: adult education, distance education, lifelong learning, qualitative research
Introduction
Education is one of the ways to promote individual’s capabilities, autonomy, increased

participation in society and sustainable development. The right to education is not limited to
children and youth. It also encompasses adults in different levels of learning, including higher
education. Thus, DE is becoming part of educational policies aiming at increasing access to
education, as many adults seek to continue their studies but due to different reasons, they are
not able to participate in on campus-education. DE delivery in an approach of online education
is a crucial learning opportunity “as it is easily accessible and facilitates close communication
between teachers/tutors and students” (Carlsen et al., 2016, p. 28). Some of the reasons of
increased demand for DE is the need of increased participation in society as a consequence of
improved competencies and critical thinking. In this context, distance education modality is
becoming one of the ways to help youth and adults to meet their rights. However, it is evident
that DE “constitutes a barrier for who do not have access to equipment and internet” (Carlsen et
al., 2016, p. 28). Given its important role in learning, DE must be accessible for all, including
adults.
The role of adult learning and education is recognized in the Agenda 2030 for Sustainable
Development. For instance, in the context of education, the goal four of this Agenda seeks to
“ensure inclusive and equitable quality education and promote lifelong learning opportunities
for all” (United Nations, 2016, p. 5). This purpose implies increased innovations and flexibility

Alzira MANUEL, Domingos BUQUE, Rosário QUIVE. Students’ perceptions on distance education: A case study in Mozambique
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
230 in the field of education. The traditional way of education and learning (classroom learning)
is no longer enough to satisfy the needs of youth and adults as well as the increased demand
for education. Therefore, DE is becoming a worldwide essential modality of education and
it demands that people change their practices. That is, students and teachers must be keen to
experience ‘changing pedagogies’ (Carrington & Robison, 2009).
However, DE in the context of selected Mozambican university is characterized by
different challenges (Brito et al., 2017; Mombassa & Aruda, 2018) which can impact the quality
of the program and concur for demotivation and drop out of the students. In this sense, the
purpose of this study is to get insight of the youth and adult students’ perceptions of DE on their
learning process.
Research Problem
Distance Education is considered a modality that contributes for the increased educational,
social, economic, and cultural inclusion in society, particularly in the case of Mozambique
(Mombassa & Aruda, 2019). It is one of the strategies for the reduction of inequalities, as
classroom learning would not be able to provide learning opportunities to many people, mainly
in rural areas (Mombassa & Aruda, 2018), where around 70% of population live, according
to Instituto Nacional de Estatística - INE (2019). It is evident that the rapid development and
expansion of Information and Communication Technology, and its application in education
in Mozambique, increased the number of beneficiaries of DE (Mussa, 2010), particularly in
higher education. DE is considered a modality that ensures access to different levels and types
of quality education to many people (Universidade Eduardo Mondlane, 2013). It is also a
modality that helps the inclusion of those who are not absorbed by the on-campus education
modality of public higher education, due to its lack of capacity in developing countries, such as
Mozambique. Despite the increased expansion and relevance of this type of education, different
constraints seem to interfere in its goals.
In fact, constraints affecting learning programs have been pointed in different studies in
online and classroom learning in Mozambique and other developing countries. In Mozambique,
Mombassa and Aruda (2018) refer to the need of teachers’ competencies improvement in DE
at universities and design of a national policy that can promote this modality. Universidade
Eduardo Mondlane (2013), amongst different challenges in DE, in this institution, points out
pedagogical, organizational, and technological aspects impacting the quality of this learning
modality. In their turn, Manuel et al. (2015) in the study about non-formal vocational education
and training in Mozambique, amongst other aspects, concluded that pedagogical aspects,
including the improvement of teacher’s competencies and administrative aspects in these
programs need to be improved.
Similarly, the study carried out in South Africa, (Baijnath & Butcher, 2015) quoted by
Prinsloo (2019), concluded that the great challenges of online DE in South Africa include “the
infrastructures and limited bandwidth, political and cultural factors, student access to devices, a
commitment to new learning model and developing staff capacity” (p. 77). Regarding cultural
facts, Muiru and Mukuria (2005) pointed out the existence of ‘pockets of cultural conservatism’
in Kenyan communities as a factor impeding women to fully participate in learning programs.
Accordingly, “parental preference for a wife and mother role for daughters” has a “pervasive
influence on the situation of women” (op. cit., p. 94). All these aspects may hinder the main
objective of education, in this case, the objectives of DE that are linked to the promotion of
inclusive quality education towards social justice, change and development.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Research Focus 231
DE is becoming an important modality of education in the world. Its characteristic of

flexibility helps many youth and adults who cannot participate in on-campus education. In this
sense, DE contributes to the increased access to education, promoting in this way, educational
inclusion, especially in the context of developing countries, where access to education through
classroom modality is still a great challenge for the governments. According to Musingafi et
al. (2015, p.59), “Zimbabwe acknowledge the importance of open and distance learning in
bringing higher education to the disadvantaged groups like women and disabled”.
DE can be defined in different ways, one of them being that it is a self-directed, autonomous
learning experience, based on technology that connects learners with resources, instructors, out
of formal institution (Johnston, 2020). However, a single definition cannot embrace all the
experiences of this modality of education. DE can also be defined as an instructor-led learning
experience employing technology to expand the synchronous classroom experiences across
distances (Johnston, 2020). While the first definition suggests high autonomy and flexibility
amongst students, the second one underlines the existence of an instructor as facilitator of the
learning process, with the learners at the heart of it. The major advantages of DE, particularly for
youth and adult learners are linked to its characteristics of “flexibility, accessibility, affordability
and life-based education opportunities” (Musingafi, 2015, p. 59).
Research reveals that “the trends in the field of DE shifted in pedagogical perspective
and theoretical frameworks with student interaction at the forefront become beating heart of
learner-centered strategies and environment” (Carlsen et al. 2016, p. 49). The learning methods
in DE open up the possibility for students to “start studies whenever they want” (or they can)
(Carlsen, et al., 2016, p. 28). This aspect is a benefit for adult learners who are often involved
in different responsibilities, which can constitute a barrier for their continuing learning process.
This advantage turns DE into a fundamental part of higher education (Tucker, 2001), mainly
in recent era where information and technology are becoming essential elements for social and
economic life.
The increased technological development that influences the world demands increased
need of education, and simultaneously introduces profound changes on it. According to the
Strategy of Distance Education, in Mozambique (Universidade Eduardo Mondlane, 2013) the
progress of DE is linked to the challenges brought by the rapid development of technology
that allows the creation of virtual environments for the increased interaction between different
actors of the education process. DE is one of the strategies of promoting learning amongst
adults, which is essential in nowadays context of rapid changes and development. Currently,
in the context of Mozambique, DE emerges as an opportunity to expand access to education,
particularly higher education (Mombassa, 2019).
According to UNESCO (2010), adults who prolong their education have great access
to information and knowledge needed to improve their work and their life, contributing for
continuing positive changes, at individual, familiar and social dimensions. This demonstrates
the power of lifelong learning, with learning not as a commodity but as a way to enrich people’s
biography (Jarvis, 2007).
Theoretical Considerations
Lifelong Learning in the Context of Adult and Continuing Education
Lifelong learning is a perspective that contributes for the increased social, cultural,
economic, and political participation in society. Learning is related to changes and changes
require learning throughout individual’s life. In this sense, London (2012, p.1) explains that

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
232 “change imposes gaps between what is and what is going to be, or between what was and what
is now”. It is this difference that moves people to engage in learning activities. Otherwise,
they “are likely to be mired in the past, perhaps struck alone on a plateau other move away
and ahead…” (Idem, p. 2). Learning promotes social inclusion, as the life in society is always
changing, and people need to be aware about how they must behave, what they need to know,
to ensure their participative citizenship.
According to Jarvis (2007, p. 5), “in contemporary society the life-world changes so
rapidly (…) it is never static …” thus, people need frequently to learn to be able to adjust
themselves to the changes and to promote transformations, and “find their place in society”.
Learning creates new competencies and opens new possibilities and new opportunities, which
could not be tangible without the needed capabilities. These capabilities can be developed in
different contexts according to the concrete needs of each individual or group of individuals.
Thus, lifelong learning encompasses all types of educational and learning experiences, for
adult, aiming to improve their knowledge, skills, and critical thinking abilities (Ahmed, 2014).
In this perspective, “lifelong learning or continuous learning is often viewed as the
domain of adult or continuing education” (London, 2012, p. 2). Lifelong learning is also
considered a way of opening up opportunity for individuals to continually learn “to enhance
their knowledge and skills in order to address immediate problems and to participate in a process
of continuous vocational and professional development” (Idem, p. 3). For instance, Kasworm
(1993, p. 412) noted that “Adult participation in higher education has come to symbolize one
important aspect of the learning society” and sustains that there are many reasons that motivate
the necessity for adult access and participation in higher education, among which, “the need for
significant expansion of educational opportunities for adult due to the short viable life of current
knowledge…”, that affects all societies in the umbrella of globalization.
General Background
The present research is qualitative and exploratory, focusing on perceptions and

experiences of youth and adult learners in distance higher education in the chosen university,
on the period 2016-2019. “Qualitative research tends to be associated with small-scale studies”
(Denscombe, 2003, p. 233).
Research Sample
For the selection of respondents, purposive sampling was used. This sampling procedure
is not strictly prescribed, and the researcher selects the respondents according to the objective
and the formulated research questions. According to Creswell and Plano Clark (2011), purposive
sampling implies the identification and selection of respondents or groups of respondents that
are clearly familiar or experienced with a phenomenon under study. In this study, the sample is
composed of twelve students, being six from the 3rd and six from the 4th years of the undergraduate
program in Organization and Management of Education, in DE modality. Amongst them, seven
are males and five females. The selection of the students from two different levels was for the
purpose of triangulation of their perceptions.
Instruments and Procedures
This study used semi-structured interview that was applied to the selected students. The
data collection was carried out in November, December 2019, and February 2020, before the

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
spread of COVID 19 pandemic in Mozambique. Thus, it was possible to perform face-to-face 233
interviews, with the respondents. The first contact with the students was by phone and it served to
schedule the interview. Then, semi-structured interview was applied to collect their perceptions
and experiences about their learning process and context. The selection of the respondents was
also dependent on their geographical location and availability, as most of them live far from
the university. The interview was carried out and analyzed in Portuguese and only the excerpts
used in the results section were translated into English, as quotations. The translation of the
respondents’ sayings from Portuguese into English can be considered one of the limitations of
this research, as the original words may have been affected.
Data Analysis
After data collection, the interviews were transcribed verbatim. According to Denscombe
(2003, p. 183), the transcription of data is “a very valuable part of research because it brings
the researchers close to the data”. In fact, the transcription process during this study allowed
getting in mind the different ideas expressed by the respondents and to understand their focus
before data analysis. The data transcribed were read repeatedly to understand their meaning,
and content analysis method was used. In this sense, after the process of familiarization with
the data, these were interpreted, and similar information was grouped and coded. Bengtsson
(2016), explaining how to perform qualitative study using content analysis, advocates the need
of using the coding list and clarification notes of the codes to reduce a cognitive change during
the process of analysis. This procedure contributed for trustworthiness, which is fundamental
in research. Later, the grouped data and their codes were interpreted and some of the data were
reduced, taking into account the objective of the study. Then, the categories emerged.
Ethical Considerations
To meet ethical procedures, in the first contact with the students, they were given
explanation on the purpose of the research and asked about their willing to participate in it.
Later on, the interview schedule was negotiated with them and the interview was carried out
at the university. Aiming at avoiding any personal harm, the names of the respondents were
replaced with a “R” (Respondent), followed by a number according to their position in the list
of interviews.
Research Results
The data collected and analyzed conducted to the following categories: General
perceptions about Distance Education, Individual, professional and gender division of labor,
Pedagogical and Organizational issues.
General Perceptions about Distance Education
The students from 3rd and 4th year included in this study perceived DE as a useful modality
of education that contributes to expand access to education. In this perspective the R1 pointed
out that “it allows people to study wherever they are…as long as there is internet signal”
and allowed conciliation of studies with employment as “there is no need to go to university,
[having to go there] only in the period of examinations”.
Apart from the perceptions linked to increased access to education, R2, understanding
the increased role and importance of Information and Communication Technology (ICT),
considered DE as a way to improve people’s competencies in this matter, while interacting

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
234 in the learning program. This modality is also considered one of the ways of improving
communication and negotiation skills. For instance, R8 noted that “many times it was necessary
to negotiate with the lecturers the schedule of the classes and assessments” and “the interaction
during the class, improves the capacity of discussing important topics respecting the ideas of
others”. R4 saw the pedagogical advantage of DE as he noted that “Distance Education allows
the implementation of student-centered learning approach (…) it ensures that students prepare
their lessons and participate actively in the learning process interacting with the lectures and
the colleagues”.
The perceptions that DE contributed to improve different competencies were repeatedly
registered in this research. For instance, R1 noted that DE improved some psychological
capacities such as “sense of responsibility and the interest in reading”. In summary, according
to this research, DE contributes for increased access to education, particularly adult education,
improves student’s competencies in ICT and in negotiation skills and promotes student centered
learning approach.
Individual, Professional and Gender Division of Labor
The respondents in this study noted that sometimes they did not have enough time to
carry out their tasks in the context of their studies due to the demand of work in the schools
they taught, especially in the period of examinations. One of the respondents explained that it
happened that when he could get time to do the tasks, the deadline had expired and the system
had closed, therefore he could not send the task on time and he failed the module.
Another problem pointed out was the weak internet signal in their residential areas or
workplace. R4 explained: “due to weak internet signal I used to climb the mountain trying to
find the internet…because from there it seems easy”.
The problem of lack of internet signal was mentioned by the respondents as one of the
major challenges, as it impacted the learning process, in individual and collective way as R4
clarified: “…sometimes we did not have class due to the weak internet signal”. R6 noted that
“the internet is a huge barrier; sometimes teachers give us homework with deadlines; if at the
time of submitting the task the internet is weak or oscillating, it is not possible to send the work,
and this impacts our pedagogical results”.
Still in this category, R5 explained that most of her colleagues of the DE program at the
university were teachers in Primary Education schools. They worked at the districts, far from
their families, and they used to go home only on Fridays, to visit their families. In her words:
“on Friday I don’t have time to prepare my classes because I use to be focused on my trip back
to home”. R5 added: “on Friday I use to attend the class being in “My love” going home and
in some places, there is no internet signal...between the village and the city…”. (“My love” is a
local popular denomination of a small lorry that serves as a semi-collective bus, and passengers
travel standing and supporting each other).
The study also demonstrated that issues linked to gender division of labor within the
families also constituted one of the challenges amongst women during the learning process.
For instance, R3 noted that “If someone [woman] goes home before concluding her academic
tasks no one will understand [at home] that tasks linked to her studies must be done…”. Similar
perception was also expressed by R5, who said: “when I have many tasks with deadlines, I must
solve them at home, where I have children to take care, familiar programs to attend and the
general domestic work to carry out…all these activities on Saturday, because Sunday I must go
back to the locality where I work …this is painful.”
In summary, the respondents expressed challenges linked to the demand of work in the
schools where they taught, the problems linked to the weak internet signal in their residential
areas or workplace. Furthermore, female respondents brought other challenges related to their
socio-cultural environment of gender division of labor, impacting their learning process.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Pedagogical and Organizational Issues 235
The pedagogical issues were pointed out in different perspectives by the students from
the 3rd and 4th years. For instance, R3 noted that the way lecturers used to send the reading
material was a challenge as some lecturers, instead of sending the basic material electronically
in the platform, they only sent the topics and the respective links. According to her “it is true
that students must be able to search information in Google, but what happens is that not all of
us can easily find the material through links, due to the slowness of internet…at the district!”
In the category of pedagogical issues, the lack of feedback during the learning process
was pointed out as the respondents perceived that the feedback on formative assessments was
almost inexistent. In the words of R5: “usually the lecturers do not give us… feedback about
our work, during the learning process, they only publish the preliminary pedagogical results,
and we do not know where we did wrong or why we got 10…” (The maximum classification
is 20 marks in the case of Mozambique). Apart from lack of feedback, students also noted the
delayed publication of the pedagogical results, at the end of the modules. This could negatively
impact their performance in the examination phase, as R8 said: “Some lecturers delay the
publication of the pedagogical results at the end of the module (…) sometimes we have the final
examination on Saturday and the lecturers inform the results the day before, on Friday (…)
and students who must write the examination do not have enough time to prepare themselves
effectively”.
Besides these issues, the pedagogical concerns linked to the lack of interaction between
the supervisors and the students in the phase of elaboration of the monograph were also pointed
out. For instance, R5, student from the 4th year, explained “…Him! I feel difficulties in the phase
of monograph …I have finished the curricular part in 2017, but until now [2019] I did not finish
my monograph…it looks like my supervisor is always busy…” In the same perspective, R10
noted that some lecturers imposed the need of hard copies of the text of monograph during
the process of supervision while the course was at distance, as he said: “…the classes are at
distance, but some lecturers require the supervision to be done face-to-face and the monographs
to be sent in hard copy… This is difficult for the students as they must print out the text of their
monograph several times and have to travel for supervision…”
In this study, the organizational concerns were also perceived in terms of slowness of
response to the student’s requests. For instance, R12 revealed: “I…I have submitted documents
to get the certificate of the modules that I have concluded, a year ago, until now it is not ready”.
In this category, the respondents brought different issues, such as lecturers sending
topics and links, instead of the basic learning material. This became difficult for students to
access those links due to the weakness of the Internet signal, lack of feedback related to the
formative assessment, which had influence in students’ progress and development, delayed
information of the results before the examination, leaving students without enough time for
their effective preparation, lack of interaction between the supervisors and the students in the
phase of monograph (this is one of the aspects that contributed for the delayed conclusion of the
course) and compulsory face-to-face supervision, when the classes were on distance modality.
Discussion
DE is seen by the respondents of this research as a helpful modality of education that

contributes to expand access to education to many youth and adults who otherwise would not
get opportunity to participate in learning programs, particularly higher education in rural areas.
In the context of Mozambique, in rural areas there are no institutions of higher education.
These institutions have their premises in big cities. This constitutes a source of exclusion for
many youth and adults, who work and live in rural areas. In this context, DE opens up new

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
236 opportunities for people from rural areas, and many other citizens, becoming, as Tucker (2001)
concludes, a very important part of higher education.
The respondents of this study perceive DE as contributing for the improvement of
people’s competencies on Information and Communication Technology (ICT), an essential tool
for life in the 21st century, in which “people are always becoming” (Jarvis, 2010, p.39). In
fact, during the learning process, through interaction with technology, instructors and learning
content, students have an opportunity to improve their knowledge, skills, and positive attitude
in relation to ICT. Apart from these perceptions, it was noted that DE, due to its teaching and
learning methodology, which demands the involvement of students as subjects of their own
learning process, promotes student centered learning. This learning methodology is a basis for
the development of autonomy and critical thinking, which are important requisites for life in
the context of rapid social, economic, political, and technological changes that characterize all
societies.
According to Jarvis (2007, p. 17), “society is changing so rapidly that many of the
traditional education organization (and its traditional methods) are not able to keep abreast with
the new demands…” In this context, the sense of responsibility is also developed in the context of
DE given that the students, as youth and adults, have enough space to control their own learning
process. However, DE in developing countries context, as it is the case of Mozambique, is
embedded in many different constraints, which can hamper its quality and relevance. This study
points out constraints linked to the students themselves, constraints brought by their profession
and constraints linked to socio cultural issues within their families (gender division of labor) and
pedagogical and organizational constraints. Individual and professional constraints are linked
to the fact that the respondents of this study are also teachers in primary schools and regularly
they have to apply assessments to their pupils, and at the end of the year, obviously, they have to
apply and mark exams. Thinking that in the Mozambican context, in public education, classes
of primary and secondary schools are composed of around thirty to sixty pupils, it is obvious
that professional issues interfere in the learning process of student teachers.
One of the most pointed aspects that constitute a big challenge in DE is the weak internet
signal in the villages, where the students work and live during the week (they live and teach at
the district, but all of them come from the city and they go back on Friday). Internet has been
one of the factors that impede students to comply with the deadlines for assessment submission.
Consequently, some of them must repeat the assessment in some modules or simply they fail.
Issues linked to gender inequality, gender division of labor were noted as part of the
challenges for women students, within their families. Some families do not understand that
women, just as the men, have the need and the right to improve their capacities through formal
education regardless of being adult, married and mothers. These families constitute ‘pockets
of cultural conservatism’ (Muiru & Mukuria, 2005). In this sense, female students face great
challenges when they need to cope with their academic tasks during the weekend at home. They
perceive that they have many domestic tasks to carry out and sometimes they are not able to meet
the deadlines for submission. This situation affects their performance as students, contributes
for the delayed completion of the course and can contribute for student’s demotivation and drop
out.
In this study, pedagogical and organizational concerns were also pointed out. Students
perceive that during the learning process their lecturers must provide them with feedback. In
whatever learning process, feedback is of great importance as it is part of formative assessment
and instructors, as facilitators, must provide each student with this information. Feedbacks
during the learning process allow students to understand what they know and what they need
to improve. Thus, feedback is one of the requisites for competencies development during the
learning process.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
In the same perspective, students also perceive that in some modules the pedagogical 237
results are announced belatedly, sometimes only one day before the examination. This result
is similar to the study carried out in Zimbabwe (Musingafi, et al., 2015, p. 65) that pointed, as
instructional related challenges “ineffective and delayed feedback of students’ assignments…”
This constitutes a constraint for those students who need to prepare themselves for the
examination and can be one of the factors for low pedagogical results and repetitions. In this
matter, lack of interaction between some students and some supervisors was pointed out as one
of the reasons for the delayed conclusion of the program. It is possible that some supervisors
are busy with other tasks at the university, and the students, being teachers themselves, use to be
engaged in many teaching activities at school lacking time to carry out their tasks as students.
However, some of the students who have interaction with their supervisors see that while
the classes are through distance modality, the supervision is planned to be face-to-face and the
monograph sent in hard copy. Clearly, there is a need of ‘changing pedagogies’ (Carrington
& Robison, 2009) among these lecturers/supervisors. This practice constitutes a barrier for
some students since they live and work far from the university, and the cost of the trip to and
from campus as well as the printing of their work for discussion with the supervisor is high
and sometimes the students did not plan this additional cost. If it is argued that DE is made
feasible at a distance by means of media which can cover extensive distances (Johnston, 2020),
students and supervisors need to negotiate strategies of interaction that can be practical for both
to allow quality of their work and a smooth interaction. The study also noted that the provision
of documents, such as certificates, comes as a challenge to the students; they must wait almost
a year when they request their certificates. This can be a signal that, nevertheless the different
efforts towards changes, there is a need of improved communication between the different actors
that collaborate in DE and an improved coordination towards quality services and effectiveness.
Similarly, Mombassa and Aruda (2019), in the study about DE in Mozambique, remarked that
the challenge of DE is in changing mentality of all university community towards an effective
DE. Musingafi et al. (2015) in the study ‘on open and distance learning’ came to the similar
conclusion as they noted that this modality has “inadequate academic support and administrative
services. In spite of some similarities with the results of the consulted studies, this study brought
some novelties as, for instance, the understanding that sociocultural issues linked to unequal
division of labor within households has a negative impact in the performance of women in
education, in DE, nevertheless it is seen as the modality that can promote educational inclusion.
This study, despite the size of the sample, provided a deep understanding of the
perceptions of students in DE relatively to their learning context and process.
The purpose of this research was to get insight of the perceptions of youth and adult
students of DE at Eduardo Mondlane University, about their process of education. The research
concludes that DE is seen as an important and relevant educational modality that contributes
for the provision of educational opportunities for many youth and adults, especially in higher
education.
DE allows participation in educational programs irrespective of the place people are.
This is an added value as many youth and adults work and live far from the cities, where the
universities are located, and they need to continue learning to improve their competencies,
particularly in this context of rapid changes in which learning is becoming part of life.
Nevertheless, the positive perception of DE, this modality is also embedded in different
constraints that can interfere in its quality, as they impact the students’ active participation
and effective learning. This study noted that the demand of work in the schools where the
respondents are teachers negatively impacts their process of learning as DE students. This

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
238 means that in their context of work they do not have enough space to negotiate their schedule
considering their studies. This situation may be a result either of teachers’ shortage, in schools
or of unbalanced power relations between the teachers and the school managers, interfering in
their work relations.
The weak internal signal in some regions impacts the learning process particularly in
relation to the process of searching the learning material. Apart from these challenges, the
study revealed that external factors such as the socio-cultural environment in the households
characterized by gender unequal division of labor, in which females are fated to carry out all
household tasks, interfere in the learning process and progress of women in DE. The study also
noted that pedagogical and organizational issues need to be improved, particularly in relation to
feedback provision to the students after the formative assessments.
Lecturers must provide feedback related to formative assessments to help students to
understand where they are or what they know and where they must arrive or what they need to
improve. Feedback contributes for the students’ self-confidence and autonomy. Therefore, it is
relevant in the learning process. Similarly, interaction between supervisors and students in the
phase of monograph writing should be active to allow students to conclude their studies within
the planned period. However, this issue is also dependent on the students’ competencies, interest,
availability, and flexibility. Finally, the study noted that coordination and communication
between administrative sector and pedagogical need to be improved to offer quality and flexible
services to the students.
In summary, the study concludes that DE is a way to provide new learning opportunities
to people, particularly young and adult people who are, in general, involved in other activities
or who live far from the cities and are not able to participate in on campus learning modality.
However, the study brought challenges that are perceived by the selected students. These
challenges can be categorized in external and internal challenges. Internal challenges are
considered all those located outside the university context. They are related to individual,
professional and gender division of labor; and internal challenges are considered those linked
to pedagogical and organizational issues, within the learning process and within the university.
References
Ahmed, M. (2014). Lifelong learning in a learning society: Are community learning centres the vehicle?
In G. Carbonnier, M. Carton, & K. King (Eds.), Education, learning, training: Critical issues for
development (pp. 102-125). Brill-Nijhoff.
Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus
Open, 2, 8-14. https://doi.org/10.1016/j.npls.2016.01.001
Brito, C., Mondjana, Ana Maria, Santos, L., & Khan, M. (2017). O ensino à distância (EaD) na
Universidade Eduardo Mondlane (UEM): Situação actual e desafios. [Distance learning (DL) at
Eduardo Mondlane University (EMU): Current situation and challenges.] Universidade Eduardo
Mondlane (UEM). https://pdfslide.tips/documents/o-ensino-a-distancia-ead-na-universidade-
eduardo-solucao-de-curto.html
Carlsen, A., Holmberg, C., Neghina, C., & Owusu-Boampong, A. (2016). Closing the gap: Opportunity
for distance education to benefit adult learners in higher education. UNESCO. https://unesco.org/
ark:/48223/pdf0000243264
Carrington, V., & Robinson, M. (2009). Digital literacies: Social learning and classroom practices.
UKLA/SAGE.
Creswell, J., & Plano Clark, V. (2011). Designing and conducting mixed methods research (2nd ed.). Sage
Publications.
Denscombe, M. (2003). The Good Research Guide for small-scale social research projects (2nd ed.).
MacGraw-Hill Education.
INE (2019). Resultados definitivos do Censo 2017. IV recenseamento geral da população e habitação
[Final results of the 2017 Census. IV general population and housing census]. Instituto Nacional
de Estatística.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Jarvis, P. (2007). Globalization, lifelong learning and the learning society: Sociological perspectives 239
(Vol. 2.). Routledge.
Jarvis, P. (2010). Adult education and lifelong learning: Theory and practice (4th ed.). Routledge.
Johnston, J. P. (2020). Creating better definitions of distance education. Online Journal of Distance
Learning Administration, 23(2). https://eric.ed.gov/?id=EJ1260793
Kasworm, C. (1963). Adult higher education from an international perspective. Higher Education, 25,
411-423.
London, M. (2012). Lifelong learning: Introduction. The Oxford Handbook of lifelong learning.
https://doi:10.1093/oxfordhb/9780195390483.013.0013
Manuel, A., Popov, O., & Buque, D. (2015). Towards programme improvement in non-formal vocational
education and training in Mozambique. Problems of Education in the 21st Century, 67, 61-71.
http://www.scientiasocialis.lt/pec/node/985
Mombassa, A., & Arruda, E. (2018). História da Educação à distância em Moçambique: Perspectivas
actuais e as contribuições do Brasil. [History of distance education in Mozambique: Current
perspectives and contributions from Brazil]. Praxis Educativa, 13(3), 643-660.
Mombassa, A., & Aruda, E. (2019). Educação à Distância em Moçambique: Perspectivas para a inclusão
da população no ensino superior. [Distance education in Mozambique: Perspectives for the
inclusion of the population in higher education]. Revista Diálogo Educação, 19(61), 899-919.
Muiru, J., & Mukuria, G. (2005). Barriers to participation in Adult Literacy Programs in Kenya. Adult
Basic Education, 15(2), 85-102.
Musingafi, M., Mapuranga, B., Chiwanza, K., & Zebron, Sh. (2015). Challenges for open and distance
learning (ODL) students: Experiences from students of the Zimbabwe Open University. Journal
of Education and Practice, 6(18), 59-66.
Mussa, J. (2010). Educação a distância em Moçambique: Um contributo para o “estado de arte”.
[Distance education in Mozambique: A contribution to the “state of the art”]. Universidade do
Minho. https://www.academia.edu/3348035/Educa%C3%A7%C3%A3o_a_dist%C3%A2ncia_
em_Mo%C3%A7ambique_um_contributo_para_o_estado_da_arte_
Prinsloo, P. (2019). South Africa. In Zawacki-Richter, O., & Qayyun, R. (Eds), Open and distance
education in Asia, Africa and Middle East. Springer Briefs in open and distance education. https://
doi.org/10.1007/978-981-13-5787-9-1
Tucker, Sh. (2001). Distance education: Better, worse, or as good as traditional education? Online Journal
of Distance Learning Administration, 4(4). https://eric.ed.gov/?id=EJ643442
Universidade Eduardo Mondlane (2013) Estratégia de Educação à distância 2014-2018. [Distance
Education Strategy 2014-2018]. Centro do Ensino à Distância, UEM.
United Nations (2016). The sustainable development goal report. New York. https://unstats.un.org/sdgs/
report/2016/The%20Sustainable%20Development%20Goals%20Report%202016.pdf
UNESCO (2010). Relatório Global sobre aprendizagem e Educação de Adultos. [Global Report on Adult
Learning and Education]. UNESCO.
Received: January 13, 2021 Accepted: March 22, 2021

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
240
Cite as: Manuel, A. A. M., Buque, D., & Quive, R. (2021). Students’ perceptions on distance
education: A case study in Mozambique. Problems of Education in the 21st Century, 79(2),
229-240. https://doi.org/10.33225/pec/21.79.229
Alzira A. M. Manuel PhD, Lecturer, Eduardo Mondlane University, Patrice Lumumba Av. No
(Corresponding author) 207, 1100 Maputo, Mozambique.
E-mail: alziramanuel92@gmail.com
Domingos Buque PhD, Lecturer, Eduardo Mondlane University, Mozambique.

E-mail: domingos.buque@gmail.com
Rosário Quive Master, Retired Lecturer, Eduardo Mondlane University, Mozambique.

E-mail: rosquive@gmail.com

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
STUDENT PERCEPTIONS OF ECUADORIAN 241
VIRTUAL PLATFORMS DURING THE

COVID-19 PANDEMIC
Magda Francisca Cejas Martínez
National University of the Chimborazo UNACH and University of
the Armed Forces ESPE, Ecuador
E-mail: magda.cejas@unach.edu.ec
Mercedes Navarro Cejas

Technical University of Manabí UTE, Ecuador
E-mail: mechyn4@gmail.com
Gina Silvana Venegas Alvarez

Cotopaxi Technical University UTC, Ecuador
Email: gina.venegas@utc.edu.ec
Carlos Enrique Proaño Rodríguez

University of the ESPE Armed Forces, Ecuador
Email: ceproanio@espe.edu.ec
Derling Jose Mendoza Velazco

National University of Education UNAE & UTE University, Ecuador
E-mail: derling969@gmail.com
Abstract
In February 2020 Ecuador declared a health emergency due to the Covid-19 pandemic. On-site classes
were suspended. The Ecuadorian university population does not adapt to virtual educational platforms.
The objective of the study was to analyse student perception of Ecuadorian educational platforms in higher
education during the COVID-19 pandemic. The study was quantitative. The sample was non-probabilistic
participatory, consisting of students from different universities in the Province of Chimborazo. For data
collection, a questionnaire with four study variables was applied. The results were analysed with T
student and MANOVA. There is a positive perception of students using the educational platforms. There
were significant statistical differences according to gender variables, contrary to the levels of study and
location, there is a greater sympathetic affection of students when receiving virtual classes by female
teachers. It can be concluded that the Covid-19 pandemic accelerated the adoption of online learning by
higher education institutions.
Keywords: Covid-19, virtual education, higher education, Ecuador, education platform, virtual
education, Ecuador
Introduction
The Republic of Ecuador declares a health emergency due to the spread of the disease
COVID-19. Prevention and control measures were taken to restrict meetings and mass events.
Similarly, the application of technological platforms for virtual education and telemedicine

Magda Francisca CEJAS MARTÍNEZ, Mercedes NAVARRO CEJAS, Gina Silvana VENEGAS ALVAREZ, Carlos Enrique PROAÑO
RODRÍGUEZ, Derling Jose MENDOZA VELAZCO. Student perceptions of Ecuadorian virtual platforms during the Covid-19 pandemic
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
242 (Primicias, 2020). Ecuador was paralysed by the outbreak of the coronavirus, which caused
schools and universities to adjust to online teaching. The higher education system has begun an
unprecedented experiment in online learning. The COVID-19 epidemic has not only seriously
endangered the health of individuals but has profoundly changed the way of life and values of
humanity. It has also placed humanity in a living environment of high risk and uncertainty.
König et al., (2020) emphasised that virtual education has become the centre of our
current teaching model, but this situation continues without time certain. The difficult parts
of the large-scale transition to online learning are not mentioned. Eickelmann and Gerick,
(2020), consider that virtual class schedules must be readjusted to easily prioritize teaching
online content. Universities need to consider student assignments. Especially when students are
in different areas, economic status, lack of technological resources and coordination of work
becomes very difficult to organize distantly. Grading criteria may also change, the weight of
small weekly homework assignments will increase, and the weight of large final assessments
will decrease. Many Ecuadorian universities faced difficulties in conducting online exams.
Coronavirus SARS-CoV-2
Coronavirus is a large family of viruses that can cause illnesses such as the common cold
and even severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome
(MERS). China recently broke out with a disease that has spread to the world, namely COVID-19.
The main cause has been confirmed to be a new type of virus called 2019 Novel Coronavirus
(2019-CoV) (Hayat et al., 2020). Little is currently known about the new virus. Major public
health organizations, including the World Health Organization (WHO) and the US Centre for
Disease Control and Prevention (CDC), are closely monitoring the development of the virus
and posting the latest information through their respective websites. These organizations also
issued some recommendations on how to prevent and treat illnesses caused by the virus.
Under the new 2019 coronavirus pneumonia epidemic (COVID-19), as some higher
education institutions prepare to reopen or maintain courses, many must face the challenge of
ensuring the safety of students, faculty, staff, and volunteers. Health monitoring and preventive
standards must provide precautions on how to help protect students and employees (such as
teachers, staff, and administrative personnel) and thus slow the spread of COVID-19. According
to Sintema (2020), extracurricular and non-formal online education is being developed globally
as a preventive measure. For others, however, online education has become a formal necessity
in recent weeks. The current public health emergency is pushing online education to take the
lead. For years, discussions on online education were abundant, in this sense, face-to-face
interaction is still the main form of education, while online education plays an assistance role.
Distance Learning
While the world is working hard to cope with the impact of COVID-19, the demand for
online education, online research, distance learning and online leisure activities puts educational
institutions around the world in a unique service position. Over the years, universities have
accumulated a wealth of online experience. The pandemic of disease has forced universities
to shift from general face-to-face learning to online learning (Byrnes, et al., 2020). According
to Zhou, et al., (2020), among the recommendations for the prevention of new coronavirus
infection (COVID-19), are regulations for higher education institutions to establish virtual
classes through PE. Laboratory classes become digital, webinars, lectures, practical exercises,
in a full-time format and in an electronic information and education environment. Personal
accounts of students and teachers, email. Cloud recordings and other educational technologies
that allow indirect (distance) interaction of students and teachers.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
To clarify the aspects of the distance education regime, Al awamrah (2015) stated that 243
distance learning cannot be organized without a computer in the students' home and the teachers'
home. Computer technology must be available, mainly in residences of socially vulnerable
students. For high quality organization of distance learning and comfortable communication
one should not leave home. High speed internet is needed at home. Distance activities offer
interaction to learners, enabling training, analysis, discussion, debate, and decision-making.
Underlying distance learning is the technology of the educational process. This is mainly
done using information and telecommunication technologies in an indirect (distance) or not
fully mediated way through learner and teacher interaction. It provides interactivity to remote
participants through open access channels (Internet, etc.) (Akande, et al., 2020).
For AL-Salahat, (2016), distance learning are lectures, seminars, colloquiums,
presentations with reports. Teachers choose the platforms of class organization that are optimal
for their course. These include conference calls, online broadcasts, group chats on social
networks and instant messaging, online tests, etc. At the technological level, the most widely
applied educational systems are Google Classroom, Chamilo LMS, Microsoft Teams, Moodle,
NEO LMS, Schoology, Edmodo and Blackboard Open LMS (Alfredsson et al., 2020). Within
virtual platforms there should be scientific social networks to foster personal educational and
disseminate scientific knowledge (Sun et al., 2020)
Excessive internet use by university students can alter hormone levels due to dependence
on virtual networks (Wahab, 2020). For example, when EPs are applied, the hormone oxytocin,
which is responsible for feelings of empathy, is released (Sindiani, et al., 2020). The disadvantage
is that a person loses the ability of real communication, being used to online communication.
When texting in EPs, the person does not follow the rules of grammar and punctuation. It is a
general influence of different technologies. They use poor vocabulary, emotions are replaced
by emoticons, all this negatively affects communication in the real world. But nowadays the
Internet is positive and necessary for the university student's life (Dhawan, 2020).
Research Problem
The current educational process cannot be imagined without the use of Internet resources
and connection technologies. The idea of distance learning and monitoring educational activities
for students seems very promising. Distance teaching and learning is gaining special significance
nowadays, with the development of the Internet. Students owning personal computers, the
exchange of information between teachers and students is improving. All this contributes to
the digitisation of the educational process. In the framework of the educational process, the
most effective direction for the development of distance learning can be the establishment
of educational platforms (EP) and virtual activities. But there is a serious drawback to work
through these sites, it is their extreme unpopularity with students. The lack of preparation
prevents to achieve remarkable results in the educational process, also in the evaluation process.
It is important to note that a pandemic and its consequent quarantine present an educational
challenge that must be considered. How to educate students when they are unable to attend
university presents a difficulty, but also an opportunity.
As a problematic research situation, the following questions emerged:

- What is the reality of educational platforms in university education during the
COVID-19 pandemic?
- Are there statistically significant differences in university distance education platforms
during the COVID-19 pandemic according to the variables gender, specialization, theory/
practice, university stage and student location?

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
244 - What is the impact on academic performance before and after the COVID-19 pandemic
when applying the virtual platforms of higher education students?
To answer these questions, the following study objectives were proposed:
- Research the reality of higher education educational platforms during the COVID-19
pandemic.
- To analyse whether there are statistically significant differences in the use of higher
education educational platforms during the COVID-19 pandemic, from the variables, gender,
specialization, theory/practice, stage, location of study and special needs in teaching.
- To describe student perception of virtual platforms in university education. To describe
academic performance before and during the COVID-19 pandemic.
Highlight student perception of PE in distance education. To analyse academic
performance before and during the COVID-19 pandemic. From a practical point of view, this
research highlights variables that could affect the implementation of virtual education during
the COVID-19 pandemic.
Research Design
In order to achieve the objectives of the study, research was designed based on the
quantitative method. The quantitative methods aim to obtain information that is presented
in numerical terms (Boeren, 2018). It demonstrates the participant's behaviour and reaction
to certain research activities. The main task of such studies is to obtain numerical data, the
reaction of respondents to any event. Quantitative studies analyse the opinion of a sufficiently
large number of people (large sample), and this can serve as an indicator of the objectivity and
reliability of the data obtained. The research time was 2 months. The scope of the study was
exploratory and descriptive. The exploratory scope was characterised by presenting information
on a phenomenon, including the identification of the variables to be studied. Descriptive scoping
is a methodological and technical procedure organised logically and objectively (Mellinger and
Hanson, 2016). These scopes made it possible to obtain reliable data on the phenomenon and to
describe its variables accurately.
Population and Sample
The research population consisted of the university students from the province of
Chimborazo, Republic of Ecuador during the academic year B-2020. Chimborazo National
University (UNACH) has 9842 students (UNACH, 2021). Chimborazo Polytechnic High
School (ESPOCH) has 13000 students (ESPOCH, 2021). The Autonomous Regional University
of the Andes (UNIANDES) has 700 students (UNIANDES, 2021). Non-statistical purposive
sampling was used for sample selection. Non-statistical sampling is most useful for exploratory
studies. Non-probability sampling is used where it is not possible to draw a random probability
sample, due to time or cost considerations (Cornesse et al., 2020). In non-statistical purposive
sampling, researchers choose only those deemed suitable (Dănescu and Anca, 2012). Under this
criterion, the students chosen were in their final year of university studies. The senior students
are more committed, knowledgeable, and experienced.
In the study there are people from different professions and age categories, with different
work experience, educational level, all in the same city. In this situation for the present study,
the sample allocation was relatively homogeneous in the object structure. This selection made
it possible to alternatively assess and compare the characteristics of interest to the researchers.
The presence or absence of connections between them was identified (Zhang et al., 2020). The

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
researchers designed and sent a participatory email invitation letter to students at the above 245
universities. For considerations of time, cost and preventive health measures of non-crowding
and distancing of people (Covid-19). Students who chose to participate were selected according
to the criteria of study variable, gender, major, place of study and educational level (see Table
1).
Table 1
Distribution of the Study Sample According to the Variables
Variables Category Frequency Percentage

Male 249 49.8
A. Gender
Female 251 50.2
Theoretical 302 60.4
B. Specialization
Practical 198 39.6
Headquarters 111 22.2
C. Student location
Core 389 77.8
Undergraduate 410 82
D. Educational level
postgraduate 90 18
Instrument
The current study sought to measure the reality of digital distance learning platforms
during the COVID-19 pandemic from the point of view of university students. This reality
could be described through the attitudes or attitudinal levels of the students. For this purpose,
a Likert questionnaire was designed as a data collection instrument. 5-point Likert scale was
used. The first option Always (S) with a value greater than 5. Then the option Almost Always
(CS) with a value of 4. The middle option Sometimes (AV) with a value of 3. Then the option
Never (N) with the lowest value of only 1 point (Pari et al., 2020).
Reliability of the Instrument

The reliability of the internal consistency between the questions was determined by
Cronbach's Coefficient Alpha. To test the reliability, a diagnostic pilot test was applied to 10
students. Statistical analyses were performed in software package SPSS.25. The questionnaire
comprised four variables (gender, university educational specialization, location of the student,
educational level). The reliability level of the 21 items was .892. The Cronbach's statistical
results, being in the range of .800 and .899, are highly reliable. This assures researchers to use
the study tool and trust the results of its application (Mendoza et al., 2019).
Data Analysis

For data analysis, descriptive statistics were applied using four study variables. The first
was the gender variable (VAR.A). The second variable was university educational specialization
(VAR.B). The third variable referred to the location of the student (VAR.C). The fourth variable
was the educational level (VAR.D). Student's t test and the study of standard deviation were
developed. The study allows to analyse the mean between each variable. Also, analyse the
distribution of the data from the mean. For the analysis of the data, it was considered that a
low Standard Deviation (SD) indicates that the data points tend to be close to the mean of the
data set. A high SD indicates that the data points are distributed over a wider range of values
(Jurková et al., 2020).

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
246 As a second component of statistical analysis, the MANOVA analysis of variance was
run. The MANOVA consists of a single ANOVA with several independent variables. The
test helps to answer many research questions, including the comparison of means of several
variables. The MANOVA analysis of variance provides a way to make multiple comparisons
of several population means (Zhao, et al., 2019). To perform the MANOVA test, comparisons
were made of the variances between sample means, as well as the within-sample variance of
each of the samples.
Research Results
To answer the first question: What is the reality of educational platforms in distance
education during the COVID-19 pandemic from the point of view of university students? The
statistical properties of the sample calculation provide descriptive answers about the reality of
educational platforms with 5-points of the response options set out in the questionnaire (see
Table 3).
Table 3
Descriptive Statistics
Variable Ranking Mean Deviation SD
VAR.B 4 1.1280 0.33
VAR.C 5 1.0730 0.37
VAR.D 3 1.0720 0.25
The table above shows the degree of benefiting from distance learning considering
the Covid-19 pandemic. All expressions were high, although the challenge was great in the
importance of using and benefiting from distance education. The results showed that the variable
B, (questionnaire items B.1 and B.5) were ranked among the high levels with a ranking of 4
(almost always). Similarly, for variable C (questions, C.3 and C.4) they presented a ranking of
5 (always). It can be described that the reality of university educational platforms was always
important. Important in terms of facilitating educational, economic and connectivity benefits.
For the results of variable D (question D.4) a medium level classification was obtained, with a
ranking of 3 (sometimes). This result considers that the university virtual platforms sometimes
allow the establishment of effective groups to develop activities and homework according to
the level of studies.
To answer the second question. The answer was provided using the t-test and one-way
analysis of variances (ANOVA). For variable A (gender) the results are shown in Table 4.
Table 4
Statistical Differences in the Use of Educational Platforms With Respect to the Gender
Variable
Gender Statistics themselves N t df p SD
Male 249 3.914

Statistical function 3.920 500 .0001
Female 251 4.883

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
It is evident from Table 3, visualised by the standard deviation, that there are statistically 247
significant differences in the use of educational platforms in distance education during the
COVID-19 pandemic from the students' point of view. The results showed that students felt
more sympathetic affection when they received videoconferences or virtual classes from
female teachers, as opposed to male teachers. Regarding the variable of university training
specialisation (Theory / Practice), the results are presented in Table 5.
Table 5
Statistical Differences between Theoretical and Practical Training Perceived by University
Students with the Use of Educational Platforms
Statistics
Specialization N t df p SD
themselves
Theoretical 302 4.515
Statistical function .691 500 .490
Practical 198 4.578
In Table 5, there are no statistically significant differences regarding the use of educational
platforms in distance education during the COVID-19 pandemic from the students' point of
view according to the variable specialisation either theoretical or practical. The data indicate
that all participants use educational platforms equally. For the study of the location variable, the
results are presented in Table 6.
Table 6
Significant Statistical Differences in Student Location Variable when Using College
Educational Platforms
Statistics
Studio location N t df p SD
themselves
Headquarters 111 4.090
Core 389 4.851
It is evident from the table above that there are statistically significant differences with
respect to the variable location of the students. Most of the students are in their homes or in
cities distant from the main university campus (Province of Chimborazo) during the Covid-19
pandemic. From the point of view of the students, according to the variable of the academic
headquarters, the use of distance learning educational platforms provides favourable benefits
for current university education. For the educational level variable, the results are shown in
Table 7.
Table 7
Significant Differences in the Variable Educational Level of College Students when Using
Educational Platforms during the Covid-19 Pandemic
Statistics
Educational level N t df p SD
themselves
Undergraduate 410 4.624
Statistical function .497 500 .628
postgraduate 90 4.523

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
248 Table 7 shows that there were no statistically significant differences according to the
educational level variable when using university educational platforms during the Covid-19
pandemic. According to students at both undergraduate and postgraduate level, the use of
virtual activities at the research level was effective. To analyse the variations descriptively, the
statistical data from the ANOVA calculation are presented in Table 8.
Table 8
MANOVA Statistical Calculation
Multiple Comparison
Independent
Mean Square SE p Lower Bound Upper Bound
variable
VAR.A 13900 4.100 0.753 6.8002 20.3256
VAR.B 13850 4.540 0.002 -20.3256 -68002
VAR.C 13800 4.461 0.001 6.9700 20.4506
VAR.D 14100 4.580 0.000 7.2220 21.4450
Table 8 shows that the mean scores of the gender variable (VAR.A) were statistically
significant when compared to the other three variables with p = 0.753. The variables of
specialisation (VAR.B), student location (VAR.C) and educational level (VAR.D) were not
statistically significant (p < .0005).
Question 3 states: What is the impact on the academic performance of university students
before and after the COVID-19 pandemic when applying the virtual platforms? The statistical
data are presented in table 9.
Table 9
Differences between Average Scores in the Use of Educational Platforms in Distance
Education and their Impact on Academic Achievement Before and After the COVID-19
Pandemic
Academic Statistics
N t df p SD
achievement themselves
Before COVID-19 500 1.040
After COVID-19 500 1.149
It is evident from Table 9 that there are statistically significant differences. On the scale
of educational platforms in distance education during the Covid-19 pandemic from the point of
view of university students. The formative specialisation variable items B.2, B.6 and B.7 gave
the results "almost always facilitates academic performance through virtual activities", as well
as "levels of comfort and satisfaction with the evaluative results obtained". The student location
variable was developed through items C.2 and C.5. Item C.2 had the lowest average value
with a score of 3 (Sometimes). In comparison to item C.5 with an average value of 4 (Almost
always). This shows that students have a greater ease of learning in virtual education, compared
to face-to-face mode.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Regarding the level of undergraduate and postgraduate studies, the students with item D.3 249
all of them stood out with a rank of 5 (Always). This demonstrates the usefulness of applying
and owning videoconferences, activities and courses recorded on the platform. In the same way,
for item D.5 the students reported that at research level, with average 5 (Always) to be able to
develop in an easy and accessible way the research activities by means of the university virtual
platform. This indicates that the Covid-19 pandemic has effectively contributed to increasing
the virtual academic average.
Discussion
From the students' point of view the use of educational platforms during the COVID-19
pandemic is generally positive. However, statistical results showed that there are significant
differences in the use of educational platforms in distance education during the COVID-19
pandemic with respect to gender affectivity (male/female) in favour of females. For Bullough
(2015) male teachers in basic elementary and high school education are of low frequency
compared to higher or university education. These differences permeate the educational
mind and consciousness; hence students expect to receive educational activities from female
teachers. In Ecuadorian culture as in other countries, gender differences, beliefs and educational
preferences are found, in this order. Ahmad et al., (2019) state that similarities between male
and female teachers were found to be important. For the present study, university students in
the Province of Chimborazo showed more sympathetic affection when receiving classes or
educational guidance from female professors.
For students, with respect to the reality of university educational platforms during the
Corona COVID-19 pandemic, there are no statistically significant differences. The qualifying
ranking of high reliability and accessibility to an educational platform is determined by Waller,
et al., (2020). The researcher indicates that online education allows the teacher and students
to set their own pace of learning. It also allows for the added flexibility of setting a timetable
that suits everyone's schedule. Regarding the variable of training or educational specialisation,
whether theoretical or practical, it was established that all students use remote educational
platforms. Ashe & Molina (2021) confirmed that the use of an online educational platform
allows a better balance between work and studies, so there is no need to give up anything.
Studying online promotes vital time management skills, making it easier to find a good balance
between work and study. Having a common agenda between student and teacher can also
encourage both parties to accept new responsibilities and have more autonomy.
Regarding the variable of the academic seat or student location, the result was in favour
of the main location, i.e., home. There were statistically significant differences on the scale of
educational platforms in distance education. The academic rate before and during the Covid-19
pandemic showed results in favour of the post-academic average evaluation. This indicates that
the quarantine has indeed contributed to an increase in the academic rate. This is consistent
with that confirmed by the study of Akande, et al., (2020) and Al awamrah, (2015). The authors
state that studying online in a university environment is a challenge to the cost of equipment
(hardware and software). Although it may be a one-time purchase to acquire hardware, the
university does not provide budgetary funds to set up the technology at home.
Instructional design must integrate the use of technology, adjust the value of skills
development, and optimise feedback opportunities across platforms. All this so that students can
gain advanced skills development and improve content knowledge through virtual education
(Olasile & Soykan, 2020). In addition to the text-based tests and exams, students should also
analyse and study virtually in creative ways. Assessments should provide answers based on the
educational platform through applied didactic methods (Mirbabaie, et al., 2020 and Rapanta,
et al., 2020).

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
250 Finally, it is known that educational platforms are an important means to develop virtual
classes. Students develop skills and metacognitions that allow them to learn autonomously
and self-regulated. University EP facilitate communication. They establish a rapprochement
between teaching staff and students. This two-way communication can be synchronous and
asynchronous. But the results showed significant data in the research. The students perceive
that the virtual EP are of temporary use by the Covid-19. Students feel affective preference
for female teachers. The time factor is established. This factor establishes the balance between
work and time. By having control of time, there is autonomy in virtual learning. It was shown
that postgraduate students had better evaluative results. They had more time to carry out online
research.
It can be concluded that university educational platforms are academic communities

in which they establish various types of relationships (research, guidance, friendship, work,
education, training, internships, etc.). EP allow their members to be in continuous communication
and interaction based on a given educational dynamic. Their application in education brings
new challenges for teachers and students to find in them a new way of teaching. They also break
with traditional classroom-only teaching schemes. ICTs are a milestone in the ways of relating
and interacting, providing innovative resources for accessing information and knowledge.
However, the main challenges to overcome are access to these resources, mainly the Internet,
literacy, training in the use of the Web, the management of learning-oriented communities and
the selection of relevant technological resources.
Educational platforms break the traditional paradigms of education, allocating the
necessary time to follow the contents. They also promote the appropriate use of interactive
digital platforms. As mentioned above, the time factor helped to set the pace of study. In this
way, the students perceived that the educational platforms are very flexible in their planning.
At the evaluative level, the results before and during the Covid-19 pandemic were positive.
Especially for postgraduate students. The coordination of time allowed a better distribution of
class schedules. Flexibility in virtual education went hand in hand with convenience in learning.
In the research, university students perceived greater sympathy with classes taught by female
professors.
Online education also favoured autonomy to the individual needs and ability level of
each student. Online classes are usually shorter than conventional classes. Most of the time,
online learning platforms allow for a limited number of students at a time, which allows for
more interaction and feedback between teacher and student. In addition, there is access to a wide
range of material, such as videos, photos, and e-books. University lecturers can also integrate
other formats, such as forums or discussions, to enhance their classes. This content is available
at any time and from anywhere, offering a more dynamic and personalised education. Online
education is spreading. The Covid-19 pandemic accelerated the adoption of online education by
higher education institutions around the world.
References
Ahmad, A., Ziad, S., & Heather, F. (2019). Teaching style differences between male and female
science teachers in Qatari schools: possible impact on student achievement. EURASIA Journal
of Mathematics, Science and Technology Education, 15(12), 1-16. https://doi.org/10.29333/
ejmste/109236
Akande, O., Badmus, T., Akindele, A., & Arulogun, O. (2020). Dataset to support the adoption of social
media and emerging technologies for continued student participation. Data in Brief, 25(31), 1-7.
https://doi.org/10.1016/j.dib.2020.105926

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Al awamrah, A. (2015). Attitudes towards e-learning at the faculty of education sciences- University of 251
Jordan. Revista Europea de Ciencias Sociales, 47(1), 96-112.
Alfredsson, K., Kjellberg, A., & Hemmingsson, H. (2020). Internet opportunities and risks for adolescents
with intellectual disabilities: a comparative study of parents' perceptions. Scandinavian Journal of
Occupational Therapy, 27(8), 601–613. https://doi.org/10.1080/11038128.2020.1770330
AL-Salahat, M. (2016). Using video modeling in teaching a simple meal preparation skill for Down
syndrome students. Revista de Educación y Práctica, 7(9), 82-90. https://files.eric.ed.gov/fulltext/
EJ1095812.pdf
Ashe, S., & Molina, R. (2021). Communication in online learning – how important is it? The
Clearing House: A Journal of Educational Strategies, Issues and Ideas, 94(1), 15-30.
https://doi.org/10.1080/00098655.2020.1835795
Autonomous Regional University of the Andes, UNIANDES. (2021). Universidad regional Autónoma
de los Andes. to the height of your dreams. https://uniandes.edu.ec/ToTheHeightOfYourDreams/
quienes-somos/
Boeren, E. (2018). Methodological underdog: A review of quantitative research in key adult education
journals. Adult Education Quarterly, 68(1), 63–79. https://doi.org/10.1177/0741713617739347
Bullough, R. (2015). Differences? similarities? male teacher, female teacher: An instrumental case
study of teaching in a head start classroom. Teaching and Teacher Education, 47, 13-21.
https://doi.org/10.1016/j.tate.2014.12.001
Byrnes, K., Kiely, P., Dunne, C., McDermott, K., & Coffey, J. (2020). Communication, collaboration
and contagion: Virtualization of anatomy during COVID-19. Clinical anatomy, 34(1), 82–89.
https://dx.doi.org/10.1002%2Fca.23649
Chimborazo National University, UNACH. (2021). Figures and research. https://www.unach.edu.ec/
unach-en-cifras/
Chimborazo Polytechnic High School, ESPOCH. (2021). Transparency 2020. https://www.espoch.edu.
ec/index.php/2020-t.html
Cornesse, C., Blom, A., Dutwin, D., Krosnick, J., De Leeuw, E., Legleye, S., Pasek, J., Pennay, D., Phillips,
B., Sakshaug, J., Struminskaya, B., & Wenz, A. (2020). Review of conceptual approaches and
empirical evidence on probability and nonprobability sample survey research. Journal of Survey
Statistics and Methodology, 8(1), 4–36, https://doi.org/10.1093/jssam/smz041
Dănescu, T., & Anca, C., (2012). Opportunity and necessity in audit sampling non-statistical sampling
method. Procedia Economics and Finance, 3, 1128-1133. https://doi.org/10.1016/S2212-
5671(12)00285-7
Dhawan, S. (2020). Online learning: A panacea in times of COVID-19 crisis. Revista de sistemas de
tecnología educativa, 49(1), 5–22. https://doi.org/10.1177/0047239520934018
Eickelmann, B., & Gerick, J. (2020). Learning with digital media. Objectives in times of Corona
and under special consideration of social inequities. Die Deutsche Schule 16, 153–162.
https://doi.org/10.31244/9783830992318.09
Hayat, O., Loubna, K., Mohamed, B., Nour, E., Hamada, I., Noureddine, B., Amine, E., & Mohamed,
B. (2020). Pathogenesis of coronavirus disease 2019 (COVID-19): evaluation and prevention,
Journal of Immunology Research, 2020, 1-7. https://doi.org/10.1155/2020/1357983
Jurková, V., Žežula, I., & Klein, D. (2020). Testing on the growth curve model with intraclass correlation
structure. Statistics, 54(5), 1124-1146. https://doi.org/10.1080/02331888.2020.1811983
König, J., Jäger, D., & Glutsch, N. (2020). Adapting to online teaching during COVID-19
school closure: teacher education and teacher competence effects among early
career teachers in Germany. European Journal of Teacher Education, 43(4), 608-
622. https://doi.org/10.1080/02619768.2020.1809650
Mellinger, Ch., & Hanson, T. (2016). Descriptive statistics. Routledge.
Mendoza, D. Nieto, Z., & Vergel M. (2019). Technology and mathematics as a cognitive component. Journal
of Physics: Conference Series, 1414, 012007. https://doi.org/10.1088/1742-6596/1414/1/012007
Mirbabaie, M., Bunker, D., Stieglitz, S., Marx, J., & Ehnis, C. (2020). social media in times of crisis:
learning from hurricane harvey for the response to the 2019 coronavirus disease pandemic.
Journal of Information Technology, 35(3), 195–213. https://doi.org/10.1177/0268396220929258
Olasile, B., & Soykan, E. (2020). Covid-19 pandemic and online learning: challenges and
opportunities, interactive learning environments. Interactive Learning Environments, 28,
1-13. https://doi.org/10.1080/10494820.2020.1813180

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
252 Pari, A., Mendoza, D., & Auccahuallpa, R. (2020). GeoGebra as a technological tool in the process of
teaching and learning geometry. Communications in Computer and Information Science, 1307,
1-14. https://doi.org/10.1007/978-3-030-62833-8_20
Primicias, (2020). Ecuador declares health emergency by spreading coronavirus. https://www.primicias.
ec/noticias/sociedad/ecuador-declara-emergencia-sanitaria-por-propagacion-del-coronavirus/
Rapanta, C., Botturi, L., Goodyear, P., Guàrdia, L., & Koole, M. (2020). Online college education during
and after the Covid-19 crisis: reorientation of teaching presence and learning activity. Postdigital
Science and Education, 2, 923–945. https://doi.org/10.1007/s42438-020-00155-y
Sindiani, A., Obeidat, N., Alshdaifat, E., Elsalem, L., Alwani, M., Rawashdeh, H., Fares, A., Alalawne,
T., & Tawalbeh, L. (2020). Distance education during the COVID-19 outbreak: A cross-sectional
study among medical students in North of Jordan. Annals of Medicine and Surgery, 59, 186-194.
Singhal, T. (2020). Coronavirus review disease-2019 (COVID-19). Indian Journal of Pediatrics, 87(4),
281-286.
Sintema, E. (2020). Effect of COVID-19 on grade 12 student performance: implications for STEM
education. Eurasia Journal of Mathematics, Science and Technology Education, 16(7), 1-6.
https://www.ejmste.com/download/effect-of-covid-19-on-the-performance-of-grade-12-students-
implications-for-stem-education-7893.pdf
Sun, L., Tang, Y., & Zuo, W. (2020). Coronavirus boosts online education. Nature Materials, 19, 687.
https://doi.org/10.1038/s41563-020-0678-8
Wahab, A. (2020). Online and remote learning in higher education institutes: a necessity in light of COVID-19
pandemic. Higher Education Studies, 10(3), 16-25. https://doi.org/10.5539/hes.v10n3p16
Waller, R., Hodge, S., Holford, J., Milana, M., & Webb, S. (2020). Lifelong education, social inequality
and the COVID-19 health pandemic. International Journal of Lifelong Education, 39(3), 243-
246. https://doi.org/10.1080/02601370.2020.1790267
Zhang, L., Ray, H., Priestley, J., & Tan, S. (2020). A descriptive study of variable discretization and cost-
sensitive logistic regression on imbalanced credit data. Journal of Applied Statistics, 47(3), 568-
581. https://doi.org/10.1080/02664763.2019.1643829
Zhao, W., Zhang, F., Wang, X., Li, R., & Lian, H. (2019). Primary variable coefficient estimator for high-
dimension models. Statistics, 53(6), 1234-1250. https://doi.org/10.1080/02331888.2019.1663521
Zhou, L., Wu, S., Zhou, M., & Li, F. (2020). School's out, but class on, the largest online education in the
world today: Taking china's practical exploration during the COVID-19 epidemical prevention
and control as a example. Best Evidence in Chinese Education Journal, 4(2), 501-519.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Appendix A. Questionnaire applied to university students for data collection. 253
Questionnaire
Support from virtual platforms in cognitive learning for college students Date: ___/___/___
Instructions
Dear Student:
Carefully read each item and choose to your right (marking with X) the option you deem correct, (S) Always, (CS)
Almost always, (AV) Sometimes, (CN) Almost never and (N) Never.
Researchers undertake to submit a statistical summary of the results if it is of interest to them.
Thank you very much for your cooperation.
Options
Ítems Questions
M F
A Its gender is
S CS AV CN N
A.1 Do you feel affection when you are taught by male teachers?
A.2 Do you feel affection when you are taught by female teachers?
Theoretical Practical
B What type of specialisation does the university offer?
S CS AV CN N
B.1 Does virtual education facilitate theoretical university learning?
B.2 Does virtual education facilitate university internships?
Are discussion forums considered an important part of the
B.3
e-learning system?
Do teachers/instructors interact with students through virtual
B.4
platforms?
B.5 Do instructors provide information about electronic courses?
When receiving virtual theoretical assessments do you feel
B.6
comfortable?
Are you satisfied with the results of the virtual practical
B.7
assessments?
Are you located in the city of your university / faculty / Headquarters Core
C
headquarter?
S CS AV CN N
Does virtual education provide you with financial benefits?
C.1
(such as residence, transport, etc.)
C.2 Do you have technical difficulties connecting to the Internet?
Does the University teach you how to use the e-learning
C.3
system?
Does your university provide manuals/guidelines/instructions
C.4
to facilitate the use of virtual platforms?
C.5 Do you find it easy to learn with online education?
Is the educational platform of your university efficient and
C.6
good?
Undergraduate postgraduate
D What level of education are you in?
S CS AV CN N
D.1 Do you use the e-learning system at your university?
Are the e-courses published on the university platforms an
D.2
important reference?
D.3 Do recorded classes help you to compensate for learning?
Do you like to set up research groups to solve assignments or
D.4
homework through virtual platforms?
Do virtual platforms allow you to optimally develop papers,
D.5
reports, theses and/or dissertations?

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
254
Received: February 18, 2021 Accepted: April 04, 2021
Cite as: Cejas Martinez, M. F., Navarro Cejas, M., Venegas Alvarez, G. S., Proano Rodriguez,
C. E., & Mendoza Velazco, D. J. (2021). Student perceptions of Ecuadorian virtual platforms
during the Covid-19 pandemic. Problems of Education in the 21st Century, 79(2), 241-254.
https://doi.org/10.33225/pec/21.79.241
Magda Francisca Cejas Martínez PhD in Economics and Research, Professor, National University of
the Chimborazo UNACH and University of the Armed Forces ESPE,
Riobamba, Ecuador.
E-mail: magda.cejas@unach.edu.ec
Cejas Mercedes Navarro PhD in research, professor at the Technical University of Manabí,
Ecuador.
E-mail: mechyn4@gmail.com
Gina Silvana Venegas Alvarez Research Professor at Cotopaxi Technical University. Ecuador.
Email: gina.venegas@utc.edu.ec
Carlos Enrique Proaño Research Professor at the University of the ESPE Armed Forces.
Rodríguez Director of the Department of Languages. ESPE-L. Ecuador.
Email: ceproanio@espe.edu.ec
Velazco Derling Jose Mendoza Teacher and researcher, Dr. in education and research. National
(Corresponding author) University of Education and UTE University. Quito, Ecuador.
E-mail: derling969@gmail.com
ORCID: http://orcid.org/0000-0001-8275-3687

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
POSTGRADUATE SUPERVISION 255
PRACTICES IN EDUCATION RESEARCH

AND THE CREATION OF OPPORTUNITIES
FOR KNOWLEDGE SHARING
Patrick Ngulube
University of South Africa, South Africa
E-mail: ngulup@unisa.ac.za
Abstract
Supervisory practices are fundamental to the production of research in higher education, but these
practices are undertheorized and poorly understood. Academia needs to understand and engage with
supervisory pedagogies to reinforce their importance in knowledge production and the development of a
knowledge society. Using content analysis of master’s and doctoral research outputs completed between
2010 and 2016, this case study investigated postgraduate supervision practices in education in South
Africa, and their implication for knowledge sharing in context. The findings showed that supervision in
education has remained rooted in the classic individualistic model, with team supervision not prevalent.
The benefits of team supervision relative to the individualistic model are highlighted. Acknowledgement
of the range of competencies of supervisors and the growth of interdisciplinary and multidisciplinary
research signals the value of the team supervision model. In addition to improving research support,
team supervision also creates communicative spaces and opportunities for knowledge sharing and the
development of supervisors.
Keywords: communicative spaces, higher degrees research, knowledge sharing, postgraduate education,
postgraduate research supervision, research supervision styles, supervision models
Introduction
Postgraduate education and supervision are central to the acquisition, assimilation and
sharing of knowledge. Knowledge production is fundamental to the creation of a knowledge
society in support of knowledge-based economies (Fourie-Malherbe et al., 2016; Fullwood et
al., 2018). Currently, there are calls for postgraduate education to contribute to the knowledge
society and knowledge-based economies (National Planning Commission, 2011; Scott, 2015;
Thomson & Walker, 2010) to respond to both global and local challenges. In recognition of the
fact that postgraduate research presents communities with a competitive edge and value-adding
processes, the National Development Plan of South Africa (National Planning Commission,
2011) and the Academy of Science [ASSAf] (2010) emphasise the importance of growing
the number of postgraduate students in higher education in South Africa to enhance research
productivity and to respond to the developmental needs of the country and the knowledge
economy.
As creators and communicators of new knowledge, students and supervisors participate
actively in the knowledge society and economy. In contributing to the knowledge economy,
master’s and doctoral students depend greatly on their supervisors for research support. The
critical role of supervisors in influencing the success or failure of studies towards research
higher degrees has received considerable research attention (Bitzer, 2010; Buttery et al., 2005;
Mouton, 2001; Mouton et al., 2015). While postgraduate research depends on effective and
efficient supervision, postgraduate research supervision is both undertheorized and poorly

Patrick NGULUBE. Postgraduate supervision practices in education research and the creation of opportunities for knowledge
sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
256 understood (Buttery et al., 2005; Manathunga, 2014; Wisker, 2012). Systematic research to
determine the most effective model for postgraduate research is limited (Agné & Mörkenstam,
2018). In South Africa, postgraduate supervisory pedagogy is “a difficult and seldom explored
facet of South African academic life” (Hugo, 2009, p. 703). Academics in South Africa have
published limited research on doctoral supervision and supervisors (Mouton et al., 2015).
Equally, students in South Africa appear to lack interest in conducting research on postgraduate
supervision (Wilkinson et al., 2016). As a result, numerous gaps remain in our understanding of
research supervision in this context.
Using content analysis of theses retrieved from the electronic database of the university
under review, this case study explored postgraduate supervision practices in education research
in South Africa. The research emanated from recognition of the need to understand supervision
practices in the selected context and the implications of those practices for knowledge sharing.
Therefore, the research aimed to contribute to the literature on supervision models and the
potential contribution of these models to knowledge creation and sharing.
Theoretical Background and Conceptual Framework
Models of Supervision
Debates relating to postgraduate mentorship have placed supervision arrangements under

the spotlight (Lahenius & Ikävalko, 2014). Supervision and supervision models are notoriously
vague and there is no common nomenclature for describing supervision models. For example,
Phillips and Pugh (1994) did not differentiate between supervision styles and approaches, while
Lee (2012) preferred to refer to supervision models as approaches, and suggested five major
conceptual styles, namely functional (directional), enculturation (contractual), critical thinking,
emancipation (laissez-faire) and relationship development (pastoral). The terms in brackets are
the equivalent supervision management styles described by Gatfield (2005) and Mouton (2001).
The supervision management style or approach determines the roles that the supervisor
assumes. These roles include that of adviser, pastor, quality controller, expert guide, coach, and
broker (Dietz et al., 2006; Mouton, 2001). Supervision styles and approaches may be employed
in both an individual and a team, or a networked context. Doctoral candidates conduct their
own research under a single supervisor in a one-to-one relationship in the individualistic or the
traditional apprentice-master model (commonly referred to as the traditional British system).
By contrast, team or networked supervision (commonly termed the American system) may take
two forms. Teamwork may entail a team of two or more supervisors working with one student.
Equally, it may involve a team or cohort of students working together on a common project
under one supervisor, who may be a principal investigator assisted by a cohort of supervisors
(De Lange et al., 2011; Van Biljon & De Villiers, 2013). Owing to the methodology that was
used, which cannot establish whether the student was supervised according to a cohort model,
cohort supervision was not considered as an indicator of supervision model for the purposes of
this article.
No singular supervision model is better than the other (Guerin et al., 2015). However,
evidence suggests that supervision in many countries and disciplines still conforms to the sole
supervisor style (Manathunga, 2012). Chiang (2003) and Backhouse (2010) identified the
individualistic model as the predominant postgraduate supervision style in the United Kingdom
and South Africa respectively. On the other hand, many universities in Australia have moved
towards team supervision (Buttery et al., 2005; Robertson, 2017).
Each supervision model has its own challenges, as outlined in Appendix 1 and the
disadvantages of collaborative supervision may expose students and supervisors to various
risks. Nevertheless, it is possible to manage the risks and tensions that come with co-supervision

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
(Watts, 2010) and the benefits of co-supervision far outweigh its disadvantages (Finn, 2005; 257
Lee, 2012). In contrast to team supervision, sole supervision fosters individualism and may be
a barrier to knowledge sharing and innovation. Team supervision, on the other hand, creates a
communicative space where “learning conversation about supervisory practices” (Wisker, 2012)
may take place. Creating communicative spaces helps supervisors to learn certain supervision
practices collectively (Carter 2016). The communicative space can enable supervisors to share
their vision of quality supervision and supervision practice. Consequently, policy discourses at
many universities around the globe are advocating team supervision as best practice (Agné &
Mörkenstam, 2018; Manathunga, 2012).
Conditions Requiring a Student to have Multiple Supervisors
Despite some of the disadvantages of collaborative supervision outlined in Appendix 1,

studies have shown that in certain circumstances it may be necessary for a student to have more
than one supervisor. These include the following:
i. Enhancing the experience of the student by bringing in specialists from
various disciplines to make inputs on theoretical, methodological, and content-
knowledge matters;
ii. ensuring that supervisors monitor one another and watch the student, with the
goal of holding the principal supervisors accountable;
iii. facilitating greater access to intellectual and practical support;
iv. bridging the gap created by differences in knowledge, expertise and supervision
experience;
v. facilitating communication and oversight when either the student or the
supervisor is operating from outside the home faculty, especially in another
country;
vi. training of novice supervisors by experienced colleagues;
vii. providing supplementary expertise; and
viii. serving as an academic mobility ‘safety net’ to ensure that the student does
not remain without a supervisor when the supervisor is on extended leave,
relocates or retires (Backhouse, 2010; Grossman & Crowther, 2015; King, 2016;
Manathunga, 2012; Olmos-López & Sunderland, 2017; Phillips & Pugh, 1994;
Pole, 1998; Van Biljon & De Villiers, 2013).
Some of the eight conditions outlined above manifest themselves in one form or the
other in many research supervision environments. The fact that some supervisors opt for sole
supervision despite the potential existence of some of these circumstances in their context
implies that they are short-changing both the students and themselves, and their ability to draw
on collective experiences may be limited if they supervise students in a dyadic manner. This may
also undermine their ability to deliver effective supervisory education and best practices, to the
detriment of the students’ research experience. Individual supervision may lead to supervisors
keeping knowledge to themselves, which is detrimental to knowledge sharing and the creation
of a learning organisation (Bureš et al., 2011).
Conceptual Framework
The conceptual framework underpinning this research resulted from drawing together
various elements, including components of theories; aspects of frameworks on postgraduate
supervision in the extant literature; personal perspectives and experiences of the researcher as a
supervisor of postgraduate students; and knowledge of the context of practice. These are some
documented strategies for compiling a conceptual framework (Antonenko, 2015; Ngulube,

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
258 2020). Since the communication dimension and knowledge creation and sharing nuances
are evident in postgraduate supervision it seemed logical that communicative action theory
(Habermas, 1987) and the theory of knowledge creation (Nonaka & Takeuchi, 1995) should be
used to gain some insights into the supervision arrangements under discussion. Figure 1 depicts
the conceptual framework showing the link between knowledge sharing and supervision models
in the context of postgraduate supervision in research education.
Communicative action creates a platform for knowledge sharing based on “mutual
recognition, reciprocal perspective taking, a shared willingness to consider one’s own
conditions through the eyes of the stranger, and to learn from one another” (Habermas,
1998, p. 159). Communicative spaces in the context of supervision may be created through
working collaboratively and supervisors sharing knowledge and skills. Supervision conducted
collectively in teams has the potential to promote knowledge transfer and open dialogue about
supervision practices among the team members. Bencic et al., (2009) underscored the importance
of teamwork in knowledge sharing and how managing knowledge may give an organisation a
competitive edge. Communicative spaces create platforms for managing knowledge and the
creation of learning organisations.
Communicative spaces may be equated to the concept of ba in the knowledge creation
theory. Knowledge creation theory focuses mainly on ba as the platform for knowledge creation,
ba being the context within which knowledge is shared, created and utilised through either face-
to-face or online interaction. This interaction has the potential to create and facilitate the use of
new knowledge. Nonaka et al. (2000) likened ba to communities of practice where individuals
with a common goal share and discuss knowledge, and in so doing improve themselves and
their practice.
Figure 1
Conceptual Framework for Postgraduate Supervision and Knowledge Sharing
Knowledge sharing, on which the knowledge economy is partly based, depends heavily
on the creation of communicative spaces and ba. Communicative spaces and ba create an
avenue for frank collegial conversations that have the potential to contribute to the members
of the team becoming productive and effective supervisors. The two theories referred to above,
namely communicative action theory and the theory of knowledge creation, provided insights
into the link between creating communicative spaces or ba within the broad context of co-
supervision and knowledge sharing.

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Problem Statement 259
Knowledge sharing among postgraduate supervisors may produce graduates that could
contribute towards sustaining a knowledge economy. The supervision models adopted by
postgraduate supervisors could provide opportunities for such knowledge sharing. Supervision
models could determine the extent to which communicative spaces for knowledge creation
and sharing are created, and how the supervision load may be shared. However, little is known
about supervision practices in education research in South Africa and how these practices could
create opportunities for knowledge sharing.
Trend studies may establish the extent to which certain supervision models are prevalent
in South Africa and their implication for knowledge sharing. However, there has been limited
research on supervision practices in higher education research in South Africa. Hence, this study
considered supervision models in the context of supervision practices. In South Africa, studies
based on a variety of doctoral supervision styles include those of Backhouse (2010), De Lange
et al. (2011), Grossman and Crowther (2015), Lessing (2011) and Manyike (2017). Guerin et
al. (2015), Manathunga (2012) and Robertson (2017) examined the practice in Australia, with
Dysthe et al. (2006) and Agné and Mörkenstam (2018) conducting similar research in Norway
and Sweden, respectively, and Fenge (2012) and Lee (2008) undertaking studies in this area in
the United Kingdom. While there have been various studies on postgraduate supervision, not
much has been written on supervision models and the opportunities they provide for knowledge
sharing.
The main contribution of the research reported on in this article, therefore, lies in its
highlighting of the neglected issue of studies on trends in research supervision and the “under-
explored” (Olmos-López & Sunderland, 2017) notion of co-supervision. The research was
mainly motivated by Severinsson (2012) who called for more studies on supervisory styles and
research outcomes. The article highlights the important link between knowledge sharing and
supervision models using communicative action theory and the theory of knowledge creation
as a conceptual framework.
Building on previous research, the five research questions outlined below guided the
research:
• What are the trends in dissertation and thesis outputs at the selected college?
• Who are the most productive supervisors and what are their supervisory patterns?
• Which is the predominant supervision model in the research outputs that were
retrieved from the selected database?
• What are the patterns of supervision by novice supervisors from the selected research
outputs?
• To what extent do the prevalent supervision models provide opportunities for
knowledge sharing?
General Background
The research employed a case study design. The college that was selected as case study
(the College of Education, University of South Africa [Unisa]) was established in 2010 and
had grown to become the second largest producer of postgraduate research in the university by
2016, as evidenced by the statistics in the graduation handbooks. It offers 11 master’s degrees
by full dissertation and doctoral degrees by thesis in 30 areas (Unisa, n.d.).
There are two schools in the college: the School of Educational Studies that consists
of the Department of Adult Basic Education, the Department of Educational Foundations,

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
260 the Department of Psychology of Education, the Department of Inclusive Education and the
Department of Educational Leadership and Management; and the School of Teacher Education
that comprises the Department of Mathematics Education, the Department of Science
and Technology Education, the Department of Language Education, Arts and Culture, the
Department of Curriculum and Instructional Studies and the Department of Early Childhood
Education (Unisa, n.d.).
In addition to vested interest on the part of the researcher, education research was chosen
because education has “an ancient lineage” (Lagemann, 1997, p. 5). Furthermore, Wilkinson et
al. (2016) confirmed that master’s and doctoral studies in South Africa between 2006 and 2016
were concentrated in the same discipline. In this article, master’s level research reports were
referred to as dissertations, while those at doctoral level were referred to as theses in accordance
with the naming convention used by the university under study.
Sample Selection
Theses and dissertations completed between 2010 and 2016 from the ten departments of
the College of Education at the selected university were used in this research. The starting point
of 2010 was chosen because that is when the college was established. The cut-off date for the
analysis, 2016, exceeded the five-year span for determining the changing trends in scholarly
communication recommended in the literature (Stansbury, 2002). The two additional years were
included to allow for possible delays in uploading research outputs on the institutional repository
within a five-year span. Data was extracted from the Directory of Open Access Repositories
(University of Nottingham, 2006–2017). Coursework dissertations were not considered for this
research because they are of limited scope and do not place the same demands on a supervisor
as a full dissertation, which requires more depth and research.
For the triangulation of data sources, the college website and available graduate booklets
were scrutinised during data collection. Although Herman (2017) found the NEXUS database
of the National Research Foundation (NRF) to be problematic, the records from the institutional
repository were also compared with those in the NEXUS database. The records from the
institutional repository proved to be more comprehensive than those in NEXUS.
Following Frick (2016), a final total of 602 retrieved dissertations and theses were
subjected to content analysis. Content analysis involves the reduction of text or other media
into categories based on coding (Weber, 1990). The sampling scheme for this research was
multifaceted. The first phase involved (i) the selection of the research outputs for the education
discipline; (ii) determining the format; that is, whether the output was a thesis or dissertation;
(iii) scoping the dissertation; (iv) indicating the name(s) of the supervisor(s); and (v) indicating
the year of completion. The first sampling phase identified 620 outputs, 367 of which were
dissertations, while 253 were theses reviewed. From that, a total of 11 dissertations and seven
theses were excluded because either the year of completion, the field of study, or the name(s)
of the supervisor(s) were not specified. The number for final analysis was 356 dissertations
and 246 theses as illustrated in Table 1. This phase determined the sample that was going to be
subjected to further analysis in the second phase.
Codes were developed for the second phase of sampling. Developing codes for measuring
patterns of scholarly communication is complex and poses difficulties for many researchers.
Two coders were involved in coding the variables of supervision trends. The classification
criteria used to determine the indicators of supervision trends were: year of completion of
the research output; format (i.e., thesis or dissertation); school to which the research output

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
was submitted; year of completion; production language; name of supervisor(s); number of 261
supervisors per research output; academic rank(s) of the supervisor(s); and subject of the
research output. Cohen’s kappa (κ) (Cohen, 1960) was used to determine the level of agreement
among the coders. Kappa values of 0.40 to 0.60 are considered fair, 0.60 to 0.75 as good,
and over 0.75 as excellent (Bakeman & Gottman, 1997). The intercoder agreement between
a postdoctoral fellow who coded 36 dissertations and 25 theses independently and the author
was 0.93. The intercoder agreement for the phase of coding was also excellent at 0.97. The high
level of intercoder agreement showed that the taxonomies chosen for the codes were reliable.
Descriptive statistical methods comprising frequency and percentage were used to
summarise the results. The coded data were analysed using Excel®. The analysis distinguished
theses from dissertations; determined the most productive supervisors and the prevalent
supervision models; and established the supervision patterns of novice supervisors.
Research Results
Trends in Dissertation and Thesis Output at the College
As shown in Table 1 more dissertations (59.1%) than theses (40.9%) were produced
during the period under review. A total of 6.5% of the dissertations and 2.8% of the theses were
in the Afrikaans language, with the bulk of the research outputs being in the English language.
None of the research outputs in Afrikaans were co-supervised. The findings set out in Table 1
revealed that 4.5% of the dissertations submitted in English were co-supervised, as were 10.2%
of theses. In all, 134 supervisors supervised 356 dissertations, and 99 supervisors supervised
246 theses. Each supervisor supervised approximately three dissertations. The mean production
rate for theses was at equilibrium with that of dissertations. The mean may be used cautiously
and tentatively to measure performance in the production of research outputs at the college for
each supervisor over a rolling period of seven years.
Table 1
Trends in Research Outputs
English Afrikaans Co-supervised Total

Year
Dissertation Thesis Dissertation Thesis Dissertation Thesis Dissertation Thesis
2010 33 21 6 1 3(13.6%) 0 39 22
2011 33 22 2 1 0 2(8.7%) 35 23
2012 57 32 6 0 3(4.8%) 7(11.1%) 63 32
2013 42 42 4 3 3 (6.5%) 7(15.6%) 46 45
2014 69 39 3 0 2(2.8%) 4(10.3%) 72 39
2015 70 48 1 1 3(4.2%) 3(6.1%) 71 49
2016 29 35 1 1 2(6.7%) 2(5.6%) 30 36
Totals 333 239 23 7 16 (4.5%) 25 (10.2%) 356 246

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
262 Most Productive Supervisors and Their Supervisory Patterns

The completion rates were difficult to determine from the gathered data. What was
clear, however, was that at least a certain number of students had completed their research
programmes. A supervisor was regarded as productive if, during the period under review, they
had supervised more than three students, which was the mean score of the collected data. The
most productive supervisors and the aspects they supervised are reflected in Tables 2 and 3.
The initial intention was to designate the 10 most successful supervisors by means of
letters of the alphabet, namely A, B, C, D, E, F, G, H, I and J, but K and L had to be added
because supervisors J, K and L had each supervised five students. Consequently, there were 12
most successful supervisors instead of 10 in the end. While these designations were applied
in the case of supervisors at doctoral degree level (Table 2), the following notations were
assigned to the most successful supervisors at master’s degree level: A1, B1, C1, D1, E1, F1,
G1, H1, I1, J1, K1 and L1 (Table 3). The year in brackets in the first columns of Tables 2 and 3
indicates when the supervisor had attained the rank of professor, irrespective of whether it was
an associate or full professorship. A range of dates, for example in the case of supervisors C, E,
B1[K], and so on, designate the period during which the supervisors were not yet professors,
but were supervising at doctoral and master’s levels. This category of supervisors produced
most of the doctoral degrees (refer Table 4).
Table 2
Most Productive Supervisors at Doctoral Level
Number of H-index
Supervisor School Aspect supervised
students Scopus Google
Psychology of Education (5), Education
Educational
A (2010) Management (2), Inclusive Education 9 3(1**) 11
Studies
(1), Socio-education (1)
Comparative Education (1), Socio-
Educational
B (2010) education (3), Education Management 8 -(2**) 17
Studies
(4)
C (2011– Educational
Education Management (8) 8 4(2**) 18
2012) Studies
Educational 7(1*) with a
D (2011) Education Management (7) -
Studies professor
E (2010– Didactics (1), Curriculum Studies (5), 7(1*) with a
Teacher Education 2(0**) 7
2013) Education Management (1) professor
Didactics (1), Curriculum Studies (5),
F (2011) Teacher Education 7 1(1**) 7
Education Management (1)
7(2*) with
G (2010) Teacher Education Educational Leadership (7) doctorate - 4
holders
Educational Guidance and Counselling (4),
H (2010) 7 -(1**) 3
Studies Psychology of Education (3)
I (2010– Educational
2011) Studies
Didactics (1), Curriculum Studies (1),
5(1*) with a
J (2010) Teacher Education
Education Management (1), Natural 6(3**) 9
professor
Science (2)
Socio-education (1), Philosophy of
K (2010– Educational 5(1*) with a
Education (1), Education Management 2(0**) 6
2012) Studies professor
(3)
Educational Education Management (4),
L (2010) 5 3(0**) 9
Studies Comparative Education (1)
Notes. *Numbers in brackets indicate the instances of co-supervision and total number students involved
*** Web of Science H-index in round brackets

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Eight of the supervisors in Table 2 had guided more students than the mean graduation 263
rate of 6.75 graduates. Five of the 12 successful supervisors co-supervised their students. Only
supervisor J1[G] co-supervised with the holder of either a doctorate or a master’s degree, as
indicated by the numbers in the round brackets in the fourth column of Tables 2 and 3. The most
productive supervisors at doctoral level have a relatively high h-index with double-digit figure
(refer Table 2).
Table 3
Most Successful Supervisors at Master’s Level
H-index
Number of
Supervisor School Aspect supervised
students Scopus Google
Educational
A1 [H] (2010) Guidance and Counselling (10) 10 -(1**) 3
Studies
B1 [K] Educational
(2010–2012) Studies
Didactics (1), Education
Teacher
C1 (2012) Management (5), Curriculum Studies 9 3(2**) 7
Education
(3)
Teacher Adult Education (6), Curriculum
D1 (2010–2014) 7 - 1
Education Studies (1)
Teacher
E1 (2013–2014) Natural Science (7) 7 1(0**) 1
Education
Educational
F1 (2010–2012) Education Management (7) 7 - 0
Studies
Inclusive Education (1), Natural
G1 (2012– Teacher
Science (3), Socio-education (1), 6 - 1
2012) Education
Inclusive Education (1)
Teacher
H1 (2012) Inclusive Education (6) 6 2(0**) 8
Education
Inclusive Education (3), Education
Teacher
I1 (2010–2011) Management (2), Curriculum Studies 6 1 6
Education
(1)
6(1#)
supervised
Education Management (3), Adult
Teacher with the
J1[G] (2010) Education (2), Inclusive Education - 4
Education holder of
(1)
a master’s
degree
K1[C] Educational
(2011–2012) Studies
Education Management (2),
EducationalPsychology of Education (1), Socio-
L1[A] (2010) 5 3(1**) 11
Studies education (1), Inclusive Education
(1)
Notes. #Number in brackets indicates the total number of students co-supervised
*** Web of Science H-index in round brackets
Predominant Graduate Supervision Practices
Sole supervision practices were prevalent among the majority of the most successful
supervisors. Sole supervision was also predominant all round, with a score of 89.8% for theses

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
264 and 95.5% for dissertations as illustrated in table 3 and 2 respectively. Such a supervision
environment is likely to reduce the creation of formal communicative spaces between
supervisors and may have created a barrier to knowledge sharing.
Patterns of Supervision by Novice Supervisors
Supervisors who had master’s degrees or PhDs but had not attained the rank of professor
were considered novice supervisors. Novice supervisors were new supervisors who have not
yet mastered the art of supervision. Patterns of supervision by holders of master’s and doctoral
degrees in the College are outlined in Table 4. A total of 80% of master’s holders supervised
master’s students on their own. The situation was similar at PhD level as 83.3% of doctoral
candidates were produced by PhD holders without the benefit of co-supervision. Cumulatively,
that accounted for 14.6% of doctorates that were supervised by novices. In 2015, two PhD
holders co-supervised a doctorate.
Table 4
Patterns of Supervision by Holders of Master’s and Doctoral Degrees with Limited Experience
Doctorates produced by Master’s degrees produced Master’s degrees

Year doctoral degree holders by doctoral degree holders produced by master’s
(N=246) (N=356) degree holders (N=356)
10 (1 co-supervised with a
2010 3 -
professor)
4(1) co-supervised with a 13 (1 co-supervised by doctoral
2011 1
professor degree holders)
7(2) co-supervised with a 17 (2 co-supervised with a
2012 4
professor professor)
9(2) co-supervised with a 3(1) co-supervised with a
2013 26
professor professor
4(1) co-supervised with a
2014 6 14
doctoral degree holder
6(1) co-supervised with a
professor (2 co-supervised 1(1) co-supervised with 2
2015 professor) (1 co-supervised with
with doctoral degree holders doctoral degree holders)
doctoral degree holders)
exclusively)
2016 1 2
professor)
Total 36(6) 113(6) 15(3)
% of research
outputs produced
16.7% 5.3% 20%
with co-
supervision
Notes. Numbers in round brackets indicate the number of research outputs that were co-supervised
Discussion
There were more master’s dissertations than theses, partly due to the fact that master’s
programmes feed into doctoral programmes. Although, there are 11 official languages in South
Africa, many research outputs were either in English or Afrikaans. The submission of theses and

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
dissertations in languages other than these two in South Africa is a trend worth watching. The 265
implication is that transformation in relation to the use of the language of knowledge production
has not taken root in the college despite the fact that, nowadays, there is a move towards the
promotion of the other official languages in the production of knowledge at this level as a result
of, among other things, the move to decolonise the curriculum. For instance, the University of
Fort Hare (Feni, 2018) conferred one of the first doctoral degrees on a candidate who submitted
her dissertation written in isiXhosa, one of the 11 official languages of South Africa.
One would have expected the high performers in research supervision to be prolific
authors as a mark of the research prowess evidenced in all the major indices, including Google
Scholar, Scopus, and the Web of Science. Each successful supervisor’s research outputs
were quantified (i.e., research impact in relation to publications). High citation metrics may
suggest that the academic has had a significant impact on the relevant field, but Harzing (2007)
cautioned that that may not always be the case for a variety of reasons, including the avenues
that the authors used to disseminate their research outputs. The Google Scholar H-index seems
to suggest that the successful supervisors were making a fairly significant impact on their
field. However, as a result of the methodological limitations of the research, the international
impact made by the supervisors was undetermined. The University of Cape Town, one of the
highest-ranking universities in South Africa, requires that a supervisor have a PhD and be an
expert in their field with an international publication record (De Gruchy & Holness, 2007). An
examination of the relationship between being a successful supervisor and having a significant
impact on the field may be instructive.
There were 12 productive supervisors, but fewer than half of them co-supervised.
Successful supervisors have the potential to share knowledge with their peers and mentor
novices if a communicative space such as the one provided by collaborative supervision is
created. It is evident that supervision at the college remains rooted in the traditional apprentice–
master model. This differed from the situation in the United Kingdom, where Olmos-López and
Sunderland (2017) found co-supervision to be a common practice. Backhouse (2010) found that
styles for supervision training of doctoral degree candidates in South Africa were not uniform,
although the individualist structure was prevalent. She concluded that the tendency towards co-
supervision or a lack of it could not be attributed to any one discipline but is largely dependent
on the way that research higher degrees are funded, the number of students to be supervised
and the workloads of supervisors (Backhouse, 2010 21). Some studies concluded that team
supervision was more prevalent in the natural sciences where researchers are generally linked
to larger research projects than in social sciences (Chiang, 2003; Fenge, 2012; Grossman &
Crowther, 2015). However, owing to its scope and limitations, this research was unable to
provide any information relating to differences between supervision in the social and natural
sciences.
It is argued that the recognition and reward systems prevailing in South Africa do not
seem to support team supervision. Many postgraduate supervisors at South African universities
receive research incentives for each postgraduate student delivered, as provided for by the
South African Research Funding Framework of 2003 (Mouton et al., 2015). The number of
points awarded for promotion based on supervision is divided by the number of supervisors
who were involved in supervising a student to completion. For instance, if three supervisors
were involved in producing a postgraduate output, each would be awarded a third of the
overall score. Someone aspiring to become an associate or full professor would therefore rather
supervise a student alone than risk their scores being affected. The reasons why noteworthy
co-supervision is not practised at the college may be a variable of the research-output incentive
system. However, based on the methodology that was used, that assumption could not be made
conclusively. Further research may assist in this regard. Furthermore, funding dynamics are
known to have had profound effects on supervision pedagogy in New Zealand (McCallin &

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
266 Nayar, 2012). It may also be a barrier to knowledge sharing (Fullwood et al., 2018). Hence, the
existing incentive system is more likely to create an academic culture of individualism.
Individualism promotes the sole supervision model. The dyadic supervision model
creates a limited environment for fostering teamwork, collaboration, and networking. It does
not create a platform for sharing knowledge, as each supervisor is preoccupied with their
“secret garden” (Grossman & Crowther, 2015). By contrast, the team supervision environment
creates a communicative space, or ba, that facilitates knowledge construction and sharing
through association and interaction. Although the creation of communicative spaces, or ba,
cannot guarantee the creation of a knowledge-sharing culture, there is evidence that such
opportunities lead to a flow of knowledge and experiences, and encourage knowledge sharing
among academics (Fullwood et al., 2018).
The environment created by team supervision empowers supervisors to work
collaboratively in developing networks through which knowledge is shared. Knowledge sharing
leads to the personal development of the individual, a phenomenon that is central to learning
within societies and the learning organisation. Knowledge sharing at the college that was
investigated may facilitate innovation in supervision practices and make supervisors effective.
Effective supervisors enable their students to produce high-quality research, complete their
studies on time, and disseminate results widely. They also prepare their students for careers
in research (Phillips & Pugh, 1994). If supervisors are to improve, their knowledge needs
to deepen. Knowledge sharing has the potential to provide such an opportunity. Knowledge
sharing can be a powerful tool for academic development among novice supervisors.
The sole supervision model followed predominantly at the college also has implications
for novice supervisors. Supervision presents very real challenges for novice supervisors in
particular (Carter, 2016). Experienced supervisors should support novice supervisors so that they
could eventually become effective supervisors. Co-supervision assists beginning supervisors
in becoming familiar with the pedagogy of graduate supervision. Moreover, collaborative
supervision enables novices to learn from other colleagues. Newcomers should not be left to
reinvent the wheel. “Rather they need a practicum, with supervisors who can demonstrate,
advise, observe performance, detect errors of application, and point out correct responses.”
(Leonard, 2001, p. 42).
Grossman and Crowther (2015) recommended that novice supervisors should co-supervise
at least three theses with three different mentors with good track records before supervising on
their own. Manyike (2017) also underscored the need for experienced supervisors to share their
knowledge with novice academics. This knowledge-transfer mechanism may be beneficial to
many universities in South Africa where supervision capacity is reported to be inadequate as
postgraduate candidature is rising steeply (Grossman & Crowther, 2015). Supervisors with a
master’s degree who co-supervised with a doctoral degree holder or a professor had someone
to support them as they learnt the art of supervision. The same applied to the doctoral degree
holders who supervised with professors. It is doubtful whether it is beneficial for two doctoral
degree holders to supervise on their own without a senior colleague, as happened in 2015.
While the situation of having two doctoral holders supervising on their own has the potential for
creating a space for knowledge sharing, it may be useful for novices to learn from experienced
supervisors. Supervising with a senior colleague provides the novice supervisor with an
opportunity for capacity development, which partly explains why collaborative supervision is
also regarded as an important part of staff development (Olmos-López & Sunderland, 2017).
Novice supervisors would then be able to compare what worked for them as postgraduate
students with the knowledge and skills they would be gaining from the giants in their fields.
It is mandatory in some countries to receive additional training in research supervision
before one may supervise postgraduate students, even if one holds a doctorate (Taylor & Beasley,
2005; Wisker, 2012). In South Africa, there seems to be no formal policy governing the support

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
that new supervisors should receive through mentoring and/or co-supervision. Lessing (2011) 267
confirmed that 84% of the lecturers at Unisa’s College of Education had acquired supervision
knowledge through workshops; informal and formal mentoring; and discussion with colleagues.
The Higher Education Qualifications Framework seems to reinforce the perceived lack of a
need for mentoring arrangements, especially for doctorates, stating that: “A graduate must be
able to supervise and evaluate the research of others in the area of specialisation concerned.”
(Department of Education, 2007). However, studies have shown that simply having a doctoral
degree is insufficient to enable one to be an effective supervisor (Grossman & Crowther, 2015;
Taylor & Beasley, 2005). Where formal training is not provided, as is the case at Unisa, it is
apparent that beginning supervisors are thrown in at the deep end and teach themselves the rules
of the pedagogy of graduate supervision.
Unisa requires those supervising doctoral studies to have a doctorate: “Supervisors of
doctoral candidates must themselves hold a Doctorate and have a research record that is deemed
acceptable by the College for the appointment.” (University of South Africa, 2017, p. 8). The
interpretation of what constitutes a research record has been left fluid. That leaves each college
to use its discretion. The master’s and doctoral procedures of Unisa do not say anything about
novice supervisors supervising in collaboration with an experienced colleague, as stipulated in
other codes of practice such as that of the University of Edinburgh (University of Edinburgh,
2017).
The traditional master–apprenticeship epistemology characterised by solo supervision

was shown to be more prevalent than co-supervision at the college. Although there is no
singular correct model of supervision, the traditional model of one supervisor working on their
own with one student does not create a communicative space where supervisors are able to
collaborate and share knowledge. Although co-supervision is not without its problems, future
doctoral programmes should consider team supervision. Team supervision has the potential to
create a collaborative culture in supervision practice and develop pedagogical innovations to the
current practice, which could promote the development of professional learning communities in
a learning organisation.
The findings presented in this study may have implications for knowledge sharing and
the development of a knowledge economy. The potential of knowledge sharing should be
considered when deciding on a preferred supervision model, either in policy or in practice.
Alternative supervision models have the potential to draw on the collective experience of a
supervision team and best practices. That may lead to the increased effectiveness of supervision
in higher education and, by implication, an improvement in the quality of graduate students
produced. The research reported on this article was based on an investigation of postgraduate
supervision at a higher education institution in South Africa. Hence, it may add to the existing
understanding of supervision models and the potential opportunities they provide for sharing
knowledge and creating a learning organisation.
Lastly, the methodological limitation of this study must be acknowledged. The
study was based on a single case, and only one methodology was used. Consequently, the
significance of the results cannot be supported. The context and method, furthermore, limit the
generalisation of the findings. This article, therefore, provides only a preliminary examination
of the phenomenon, and there is room for further studies using other methodologies like mixed
methods research. For instance, no conclusion could be made about the extent to which informal
supervision arrangements were employed at the college. Informal supervision arrangements,
when effectively used, may bridge any shortcomings in the expertise of the supervisor and
lessen the supervisory burden.

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
268 However, the case study presented in this article provided insights that may apply to
postgraduate supervision in a context that follows the European or United Kingdom doctoral
model. The work should be extended to multiple colleges at the university and other higher
education institutions to provide a comprehensive picture of supervisory pedagogies in the
country and across countries and regions. The results of this research support the salience of
communicative action theory and the theory of knowledge creation in explaining knowledge
sharing in the context of research supervision. Team supervision can create communicative
spaces and ba for knowledge sharing and developing a learning organisation.
References
Academy of Science in South Africa (ASSAf). (2010). The PhD study: An evidence-based study on how
to meet the demands for high-level skills in an emerging economy. ASSAf.
Agné, H., & Mörkenstam, U. (2018). Should first-year doctoral students be supervised collectively or
individually? Effects on thesis completion and time to completion. Higher Education Research &
Development, 37(4), 669–682. https://doi.org/10.1080/07294360.2018.1453785
Antonenko, P. D. (2015). The instrumental value of conceptual frameworks in educational technology
research. Educational Technology Research Development, 63(1), 53–71.
Backhouse, J. (2010). Patterns of practice in South African doctoral education: An empirical study. Acta
Academica Supplementum, 1, 1–22. https://journals.ufs.ac.za/index.php/aa/article/view/1275
Bakeman, R., & Gottman, J. M (1997). Observing interaction - An introduction to sequential analysis.
Cambridge University Press.
Bencsik, A., Noszkay, E., & Marosi, I. (2009). Teamwork in education. Problems of Education in the 21st
Century, 10, 9-24. http://www.scientiasocialis.lt/pec/node/185
Bitzer, E. (2010). Postgraduate research supervision: More at stake than research training. Acta Academica
Supplementum, 1, 23–56.
Bureš, V., Griffin, D., Hackett, D., Kročitý, P., & Kubička, E. (2011). Rethinking of knowledge
management introduction at teaching universities: The framework development. Problems of
Education in the 21st Century, 32, 33-46. http://www.scientiasocialis.lt/pec/files/pdf/vol32/33-
46.Bures_Vol.32.pdf
Buttery, E. A., Richter, E. M., & Filho, W. L. (2005). An overview of the elements that influence efficiency
in postgraduate supervisory practice arrangements. International Journal of Educational
Management, 19(1), 7–26.
Carter, J. W., Enyedy, K. C., Goodyear, R. K., Arcinue, F., & Puri, N. N. (2009). Concept mapping of
the events supervisees find helpful in group supervision. Training and Education in Professional
Psychology, 3(1), 1–9.
Carter, S. (2016). Supervision learning as conceptual threshold crossing: When supervision
gets ‘medieval’. Higher Education Research & Development, 35(6), 1139–1152.
https://doi.org/10.1080/07294360.2016.1160875
Chiang, K-H. (2003). Learning experiences of doctoral students in UK universities. The International
Journal of Sociology and Social Policy, 23(1/2), 4–32.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological
Measurement, 20, 37-46.
De Gruchy, J. W., & Holness, L. (2007). The emerging researcher: Nurturing passion, developing skills,
producing output. University of Cape Town Press.
De Lange, N., Pillay, G., & Chikoko, V. (2011). Doctoral learning: A case for a cohort model of supervision
and support. South African Journal of Education, 31(1), 15–30.
Department of Education (DoE). (2007). The higher education qualifications framework (HEQF).
Government Gazette (No. 303530, Vol. 508). Government Printing Works.
Dietz, A. J., Jansen, J. D., & Wadee, A. A. (2006). Effective PhD supervision and mentorship: A workbook
based on experiences from South Africa and the Netherlands. Unisa Press & Rozenberg Publishers.
Dysthe, O., Samara, A., & Westrheim, K. (2006). Multivoiced supervision of master’s students: A case
study of alternative supervision practices in higher education. Studies in Higher Education, 31(3),
299–318.
Fenge, L. (2012). Enhancing the doctoral journey: The role of group supervision in supporting collaborative
learning and creativity. Studies in Higher Education, 37(4), 401–414.

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Feni, L. (2018, October 23). Doctor feels good after historic IsiXhosa PhD at Fort Hare University. Daily 269
Dispatch. https://www.timeslive.co.za/news/south-africa/2018-10-23-doctor-feels-good-after-
historic-isixhosa-phd-at-fort-hare-university/
Finn, J. A. (2005). Getting a PhD: An action plan to help manage your research, your supervisor, and
your project. Routledge.
Fourie-Malherbe, M., Aitchison, C., Bitzer, E., & Albertyn, R. (2016). (Re)Considering postgraduate
education and supervision in the knowledge society. In Fourie-Malherbe, M., Albertyn, R.,
Aitchison, C. & Bitzer, E. (Eds.), Postgraduate supervision: Future foci for the knowledge society
(pp. 1–12). SUN MeDIA.
Frick, L. (2016). PhD by publication: an institutional analysis. In Fourie-Malherbe, M., Albertyn, R.,
Aitchison, C. and Bitzer, E. (Eds.), Postgraduate supervision: Future foci for the knowledge
society (pp. 299–312). SUN MeDIA.
Fullwood, R., Rowley, J., & McLean, J. (2018). Exploring the factors that influence knowledge sharing
between academics. Journal of Further and Higher Education, https://doi.org/10.1080/030987
7X.2018.1448928
Gatfield, T. J. (2005). An investigation into PhD supervisory management styles: Development of a
dynamic conceptual model and its managerial implications. Journal of Higher Education Policy
and Management, 27(3), 311–325.
Grossman, E. S., & Crowther, N. J. (2015). Co-supervision in postgraduate training: Ensuring the right
hand knows what the left hand is doing. South African Journal of Science, 111(11/12). Article
2014-0305, 8 pages. http://dx.doi.org/10.17159/sajs.2015/20140305
Guerin, C., Kerr, H., & Green, I. (2015). Supervision pedagogies: Narratives from the field. Teaching in
Higher Education, 20(1), 107–118. https://doi.org/10.1080/13562517.2014.957271
Habermas, J. (1987). The theory of communicative action: The critique of functionalist reason. Polity.
Habermas, J. (1998). Remarks on legitimation through human rights. Philosophy and Social Criticism,
24(2/3), 157-171.
Harzing, A. W. (2007). Publish or perish. https://harzing.com/resources/publish-or-perish
Herman, C. (2017). Looking back at doctoral education in South Africa. Studies in Higher Education,
42(8), 1437–1454.
Hugo, W. (2009). Spiralling reference: A case study of apprenticeship into an academic community of
practice. South African Journal of Higher Education, 23(4), 703–721.
King, M. L. (2016). The art of joint supervision of graduate students: What advice should we give
supervisors? [Paper presentation]. 12th Quality in Postgraduate Research Conference, National
Wine Centre, Adelaide, 20–22 April. https://lo.unisa.edu.au/pluginfile.php/1090717/mod_
resource/content/1/King%202016.pdf
Lagemann, E. C. (1997). Contested terrain: A history of education research in the United States, 1890–
1990. Educational Researcher, 26(9), 5–17.
Lahenius, K., & Ikävalko, H. (2014). Joint supervision practices in doctoral education – A student
experience. Journal of Further and Higher Education, 38(3), 427–446.
Lee, A. (2008). How are doctoral students supervised? Concepts of doctoral research supervision. Studies
in Higher Education, 33(3), 267–281.
Lee, A. (2012). Successful research supervision: Advising students doing research. Routledge.
Leonard, D. (2001). A women’s guide to doctoral studies. Open University Press.
Lessing, A. C. (2011). The role of the supervisor in the supervisory process. South African Journal of
Higher Education, 25(5), 921–936.
Manathunga, C. (2012). ‘Team’ supervision: New positioning in doctoral education pedagogies. In Lee,
A. & Danby, S. (Eds). Reshaping doctoral education: International approaches and pedagogies
(pp. 42–55). Routledge.
Manathunga, C. (2014). Intercultural postgraduate supervision: Reimagining time, place and knowledge.
Routledge.
Manyike, T. V. (2017). Postgraduate supervision at an open distance e-learning institution
in South Africa. South African Journal of Education, 37(2). Art. # 1354, 11 pages,
https://doi.org/10.15700/saje.v37n2a1354
McCallin, A., & Nayar, S. (2012). Postgraduate research supervision: A critical review of current practice.
Teaching in Higher Education, 17(1), 63–74.

sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
270 Mouton, J. (2001). How to succeed in your masters and doctoral studies. A South African guide and
resource book. Van Schaik.
Mouton, J., Boshoff, N., & James, M. (2015). A survey of doctoral supervisors in South Africa. South
African Journal of Higher Education, 29(2), 1–22.
National Planning Commission. (2011). National Development Plan: Vision for 2030. Government
Printing Works.
Ngulube, P. (2020). Theory and theorising in information science scholarship. In P. Ngulube (Ed.),
Handbook of research on connecting research methods for information science research (pp.18-
39). IGI Global.
Nonaka, I & Takeuchi, H. 1995. The knowledge creating company: How Japanese companies create the
dynamics of innovation. Oxford University Press.
Nonaka, I, Toyama, R., & Konno, N. (2000). SECI, ba and leadership: A unified model of dynamic
knowledge creation. Long Range Planning, 33(1), 5–34.
Olmos-López, P. & Sunderland, J. (2017). Doctoral supervisors’ and supervisees’ responses to co-
supervision. Journal of Further and Higher Education, 41(6), 727–740.
Paul, P., Olson, J. K. & Gul, R. B. (2014). Co-supervision of doctoral students: Enhancing the learning
experience. The International Journal of Nursing Education Scholarship, 11(1), 31–38.
Phillips, E. M., & Pugh, D. S. (1994). How to get a PhD: A handbook for students and their supervisors.
Open University Press.
Pole, C. (1998). Joint supervision and the PhD: Safety net or panacea? Assessment & Evaluation in
Higher Education, 23(3), 259–271.
Robertson, M. J. (2017). Team modes and power: Supervision of doctoral students. Higher Education
Research & Development, 36(2), 358–371.
Scott, P. (2015). Higher education, the public good and public interest. In Filippakou, O. & Williams, G.
(Eds). Higher education as a public good: Critical perspectives on theory, policy and practice
(pp. 41–58). Peter Lang.
Severinsson, E. (2012). Research supervision: Supervisory style, research-related tasks, importance, and
quality - Part 1. Journal of Nursing Management, 20(2), 215–223.
Stansbury, M. C. (2002). Problem statements in seven LIS journals: An application of the Hernon/
Metoyer-Duran attributes. Library and Information Science Research, 24(2), 157–168.
https://doi.org/doi:10.1016/S0740-8188(02)00110-X
Taylor, S., & Beasley, N. (2005). A handbook for doctoral supervisors. Routledge.
Thomson, P., & Walker, M. (2010). Doctoral education in context: The changing nature of the doctorate
and doctoral students. In Thomson, P. & Walker, M. (Eds.), The Routledge doctoral supervisor’s
companion: Supporting effective research in education and the social sciences (pp. 9–26).
Routledge.
University of Edinburgh. (2017). Code of practice for supervisors and research students. http://www.
docs.sasg.ed.ac.uk/academicservices/codes/copsupervisorsresearchstudents.pdf
University of Nottingham. (2006–2017). Directory of Open Access Repositories: Unisa Institutional
Repository. http://www.opendoar.org/
University of South Africa. (n.d.). College of Education. http://www.unisa.ac.za/sites/corporate/default/
Colleges/Education
University of South Africa. (2017). Procedures for master’s and doctoral degrees. http://www.unisa.ac.za/
sites/corporate/default/Colleges/Graduate-Studies/M-&-D-policy-and-procedure-documents
Van Biljon, J. A., & De Villiers, M. R. (2013). Multiplicity in supervision models: The supervisor’s
perspective. South African Journal of Higher Education, 27(6), 1443–1463.
Watts, J. H. (2010). Team supervision of the doctorate: Managing roles, relationships and contradictions.
Teaching in Higher Education, 15(3), 335–339.
Weber, R. P. (1990). Basic content analysis (2nd ed.). Sage.
Wilkinson, A., Van Jaarsveldt, D., Grimsley, E., & Seoka, L. (2016). Research trends in higher education
studies at master’s and doctoral level: ‘Interdisciplinarity’ and postgraduate supervision. In
Fourie-Malherbe, M., Albertyn, R., Aitchison, C. & Bitzer, E. (Eds.), Postgraduate supervision:
Future foci for the knowledge society (pp. 349–363). SUN MeDIA.
Wisker, G. (2012). The good supervisor: Supervising postgraduate and undergraduate research for
doctoral theses and dissertations (2nd ed). Red Globe Press.

Appendix 1.
sharing
Relative advantages and disadvantages of various research supervision styles
Style Advantages Disadvantages
Leveraging individual skills and expertise to produce rigorous research Cautious and conforms to orthodox ideas, wary of creativity and risk taking
Do not have to carry the supervision load of colleagues Increases risk
No possibility of conflicting feedback Limited support and resource base for students
Single relationship to manage and no potential discord among supervisors Supervisor’s preferences may constrain and limit the scope and methodology of the study
Solo or
Diversity and multiplicity of views may be problematic
individual
supervision
Total commitment to the research project and full control May not be transparent
Difficult to deal with tensions in the absence of a possible mediator
The need to deal with conflicting academic identities that may give rise to personality
Shared supervisor workload and responsibilities
clashes, complex power dynamics, intellectual contests, or personal jealousies
The student has the benefit of the support of more than one supervisor to be creative

Difficulty in building trust among supervisors
and take risks by being experimental and challenging orthodox ideas
Students experience diverse supervision cultures and academic perspectives Complex, multifaceted, and dynamic, which may affect students and supervisors negatively
Development of team spirit and creation of a communicative space for sharing
Students may play supervisors off against one another
problems related to supervising students and solving problems creatively
Leveraging a wide theoretical and conceptual base (having access to a greater range
Fragmentation of supervision responsibility
of expertise)
Increased transfer and sharing of supervision knowledge than would otherwise be
One supervisor may bear the greater supervision workload, while others ride on their back
spread among supervisors
Increased cross-fertilisation of ideas and support to new supervisors (novice
supervisors have an opportunity to observe good practice as modelled by a more senior Conflicting and confusing advice and feedback to the student
colleague before taking full responsibility for supervision)
Dealing jointly with the pressure to produce a large number of postgraduate students
at the expense of quality
Potential to promote the use of various research methodologies, including mixed
methods research, by drawing on the expertise of supervisors from different research
traditions
“Insurance” or backup against illness, death, and absence or “intellectual bereavement”
Promotion of interdisciplinary interactions
Drawing on the strengths of members of the team to give superior guidance, resulting Some co-supervisors do not engage, and demonstrate only limited commitment
in a product that is informed by rigorous research
Increased throughput rate and shortened study periods
Academic relay team approach, allowing supervisors to relieve one another of
committees’ model, although the models are not mutually exclusive)

supervision responsibilities
Joint or dual or team or networked or group supervision (excludes US advisory

The other supervisors can act as mediators in the event of conflict with the student
https://doi.org/10.33225/pec/21.79.255
Notes. Adapted from Buttery et al., (2005); Carter et al., (2009); King (2016); Manathunga (2012); Olmos-López & Sunderland (2017); Paul et al., (2014); Phillips & Pugh (1994);
Pole (1998); Taylor & Beasley (2005); and Van Biljon & De Villiers (2013).
271
PROBLEMS
Vol. 79, No. 2, 2021

OF EDUCATION
IN THE 21st CENTURY
sharing
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
272
Received: September 23, 2020 Accepted: March 10, 2021
Cite as: Ngulube, P. (2021). Postgraduate supervision practices in education research and the
creation of opportunities for knowledge sharing. Problems of Education in the 21st Century,
79(2), 255-272. https://doi.org/10.33225/pec/21.79.255
Patrick Ngulube PhD, Director, University of South Africa, School of Interdisciplinary Research and
Graduate Studies, PO Box 293, 0003, UNISA, Pretoria, South Africa.
Website: http://www.unisa.ac.za/sites/corporate/default/Colleges/Graduate-
Studies/

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
THE IMPACT OF TEACHER PROFESSIONAL 273
DIARY ON THEIR REFLECTIVE,

COMMUNICATIVE AND PROFESSIONAL
SKILLS
Eka Tkavashvili
Ilia State University, Georgia
E-mail: eka_tkavashvili@iliauni.edu.ge
Abstract
Teacher professional diaries are teacher reflections on their own and their colleagues’ observed practice.
They are effective tools of increasing teacher qualification. Few research studies worldwide, while no
such research studies in Georgia have been conducted on the impact of the application of reflective
diaries on teachers’ reflective, communicative, and professional skills. Therefore, the aim of the research
was to find out whether teachers’ reflective, communicative, and professional skills developed as a result
of the application of reflective diaries, and if so, to what degree they increased. Quantitative research (an
experiment with participation of 62 teachers) was held, with an experimental group whose participants
wrote reflective diaries for 9 weeks and a control group which did not undergo any special treatment.
A 5-point Likert scale questionnaire was developed, and its reliability and validity were assessed. Pre-
experimental and post-experimental self-assessment questionnaire was held, the results were assessed and
compared through descriptive statistics and a T-test. The results revealed that there was a positive average
relationship between the application of diaries and reflective thinking, communicative and professional
skills. According to the study results, the application of teacher reflective diary can be recommended as an
effective tool for the development of reflective thinking, communication, and professional qualification.
Keywords: communicative skills, professional skills; reflection diary, reflective thinking skills
Introduction
Significance
According to Chong and Cheah (2009), “competent effective teachers help to build a
strong system of education” (p.1), such as in Singapore, second most successful in the world
country concerning general education (OECD, 2018). The quality of a teacher’s qualification,
in turn, depends on teacher knowledge, skills, and values (Chong & Cheah, 2009). Gore et al.
(2017). Antoniou et al. (2015) stated that teacher professional development had a significant
positive impact on the quality of teaching. Research by Göker (‎2016), Hilton et al. (2013),
Insuasty and Zambrano Castillo (2010), as well as Ukrop et al. (2019) confirmed that teacher
professional diaries were effective tools for teacher professional development. Göker (2016)
held a quantitative and qualitative (survey and interview) concerning teacher diaries. In Hilton
et al. (2013) study teacher diary was used as one of research tools. The participant teachers
approved of its application, as confirmed by one more research tool – questionnaire. Based on
research findings, he recommended to introduce diary writing into teacher training programs.
Insuasty and Zambrano Castillo (2010) studied the efficiency of teacher journals through
journals, blogs, metaphors, conferences, and a questionnaire. Dincel and Savur (2018) in a
qualitative study found that holding a reflective diary helped teachers to enhance their writing
abilities, professional skills and personal development.

Eka TKAVASHVILI. The impact of teacher professional diary on their reflective, communicative and professional skills
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
274 While there are numerous studies of teacher and teacher-student diary writing role in
good teaching in general (as shown above), there are fewer studies dealing with the impact of
teacher diary application on separate skills such as reflection, communication, and professional
skills. Klimova (2015) in her literature analysis claimed that teacher diaries developed their
reflection capacity. Patterson (2014) analyzed his experience of organizing a teacher diary
writing program with teacher-to-be and found many benefits of its application: developing
writing skills, self-confidence and (if held in groups) organization skills. Taqi et al. (2015)
investigated through a questionnaire the impact of journal writing on the ability to communicate
in writing and came to conclusion that communication skills were effectively developed through
teacher diary writing. The narrative-based inquiry by Aharonian (2016) and the critical analysis,
observations, and interview by Chien (2018) found that reflective writing increased teachers’
professional knowledge and skills. Lopera Medina (2013) held a case study on the application
of English teachers’-to-be diary in order to analyze the development of their reading skills. The
study found it useful for the development of reading skills.
None of the above-discussed research studies were experimental. No scholarly / empirical
research on the issue of teacher reflective diary writing in Georgia has been conducted. Only
one descriptive article by Gogoladze (2019) is available on the Internet, which explains what
reflection is and gives teachers’ practical recommendations on holding it but does not provide
any empirical data. Therefore, the current research tries to fill this gap.
Teacher Professional Diaries / Journals
Reflection helps practitioners to feel confident and competent about the matters, which
involve both personal and professional growth. A process of learning enhances their confidence
when confronted with complex and emotionally demanding situations in the workplace. Oelofsen
(2012) emphasized that teachers may gain from reflecting despite the fact that reflection is time-
consuming, while professionals have such a busy schedule (Oelofsen, 2012).
Merely reflecting on the experience does not ensure that a teacher will identify the real
causes of the problem or recognize the elements of successful teaching. ‘Critical reflection’
refers to an activity or process in which an experience is recalled, considered, and evaluated,
usually in relation to a broader purpose. It is a response to past experience and involves conscious
recall and examination of the experience as a basis for evaluation and decision-making, and as
a source for planning and action. Reflective practice is “learning through and from experience
towards gaining new insights of self and practice” (Finlay, 2008, p.1).
Reflection is more productive when concrete experience is documented in written form. This
considerably improves the quality of reflection and provides a valuable source for further
reference (Tsingos-Lucas et al., 2017). The essence of reflective diaries is to facilitate the
process of reflection by guiding a teacher through self-talk.
A diary / journal is a sequential, dated chronicle of events and ideas, which includes
the personal responses and reflections of the writer (or writers) on those events and ideas. A
journal has six defining attributes: it is written, dated, informal, flexible, private, and archival.
“Reflection is the path both to self-knowledge and to greater personal efficacy. Although there
are many ways to reflect, the journal is concrete evidence of one’s evolving thought processes,
documenting valuable, often fleeting glimpses of understanding” (Stevens & Cooper, 2009, p.
3).
Spadling and Wilson (2002) believed that reflective writing can promote reflective
thinking. They mentioned the following benefits of journal writing: (1) journals serve as a
permanent record of thoughts and experiences; (2) journals provide a means of establishing and
maintaining relationship with instructors; (3) journals serve as a safe outlet for personal concerns
and frustrations; (4) journals are an aid to internal dialogue; 5) journals serve as windows into

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
students’ thinking and learning; 6) journals provide a means of establishing and maintaining 275
relationship with students; and 7) journals serve as dialogical teaching tools (p. 1396).
Reflective journaling can be used at university level and for in-service teacher professional
education programs with a variety of purposes in focus. The efficacy of the usefulness of
reflective journals was explored while teaching writing to university students who reflected
on the instructional practices with an aim of evaluating the effectiveness of teaching (Ahmed,
2019).
Reflective diaries may be of particular importance for student and novice teachers. Diaries
guide the student / novice teachers to improve their professional skills through reflection on
teacher training sessions. In addition, diaries are appreciated and used by experienced university
teachers from various faculties and school teachers. According to Ukrop et al. (2019), it is a
“container for writing that provides students with a framework to structure their thoughts and
reflections” (p.2).
Reflective diaries are applicable for ensuring foreign language teacher development.
Reflective teaching, while applying them, is divided into four phases: 1. identification of
the problem, 2. observation and analysis, 3. re-generalization, and 4. actual verification. The
data gained through observation by means of questionnaires, recordings, interviews, etc. are
critically analyzed in reflective journals. After reflecting on their philosophy, practice, beliefs,
values, attitude and affection, teachers analyze the causes of the problem and learn from others’
experience. Through cyclic reflection, teachers become effective reflectors (Liu & Zhang, 2014).
Research Problem
Judging by students’ results (OECD, 2018), teachers in many countries (including

Georgia) do not demonstrate optimistic results in skill level. To increase the level of teachers’
skills, as shown in the literature review above, teacher reflective diaries are an effective tool.
This research is trying to fill the gap and to study how the application of teacher diary
changes three aspects of teachers’ qualification (reflective thinking, communicative and
professional skills). The impact of reflective diary application on teacher reflective thinking,
communicative and professional skills has been recently investigated to a certain degree in
various countries: in Czech Republic by Klimova (2015), in Japan by Patterson (2014), in Israel
by Aharoyan (2016), in Taiwan by Chien (2018), and by some other researchers in various
countries. However, a wider-scale / experimental investigation is needed to make far-going
conclusions. In Georgia there are no studies on the issue, so the author of the current article was
especially interested whether in Georgia teacher reflective diaries could be used for increasing
the level of teacher reflective, communicative, and professional skills.
Research Focus
Reflective diaries written by teachers can benefit them in many ways. Di Pardo Léon-
Henri (2021) enumerated the following benefits of teacher diaries:
• building understanding about themselves and their students;
• becoming a reflective professional and developing their cognitive and
metacognitive skills;
• improving observational and critical analysis skills;
• stimulating reflective thinking skills;
• developing cooperation (with the help of dialogue diaries);
• enhancing professional development;
• promoting teacher autonomy;
• helping to make decisions.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
276 This research concentrated on reflective thinking skills (first 4 items on the list),
communication with colleagues (5th item), and professional skills (2nd and 6th items), therefore,
it covered the majority of the benefits of teacher reflective diaries.
Research Aim and Questions
The aim of the research was to find out whether teachers’ reflective, communicative, and
professional skills developed as a result of the application of reflective diaries, and if so, to what
degree they increased.
The research questions, correspondingly, were as follows:
• Do teachers’ reflective thinking skills increase as a result of the application of teacher
reflection diary?
• If so, to what degree do they increase?
• Do teachers’ communicative skills increase as a result of the application of teacher
reflection diary?
• Do teachers’ professional skills increase as a result of the application of teacher
reflection diary?
Hypotheses
Null, positive and negative hypotheses were formulated:

H0a: There is no impact of professional diary writing on reflective thinking skills.
H1a: There is a positive impact of professional diary writing on reflective thinking skills.
H2a: There is a negative impact of professional diary writing on reflective thinking skills.
H0b: There is no impact of professional diary writing on communicative skills.
H1b: There is a positive impact of professional diary writing on communicative skills.
H2b: There is a negative impact of professional diary writing on communicative skills.
H0c There is no impact of professional diary writing on professional skills.
H1c: There is a positive impact of professional diary writing on professional skills.
H2c: There is a negative impact of professional diary writing on professional skills.
General Background
The scientific view, philosophy of Positivism/Realism and an empirical-analytic

paradigm constitute the research approach of this study (Cohen et al, 2018). They imply that
a quantitative (experimental) research design was used for this research as a basis for finding
out the impact of conducting a reflective diary on teachers’ reflection, communication, and
professional skills in Georgian context. This design was chosen, as Cohen et al. (2018) stated
that quantitative (especially, experimental) research design yields objective (permits statistical
treatment, correspondingly, enables generalization), valid and reliable results.
The current research was part of a larger study including defining teacher attitudes
to reflective diary writing in Georgia as its first stage (Tkavashvili, 2021) and then holding
an experiment with teachers from several schools to support the idea that the application of
reflective diaries can be an effective tool for the improvement of teacher skills. In the first
(survey) study 1046 teachers from all regions of Georgia, including private and public schools,
participated, a sample representative of the population – Georgian school teachers. The scale

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
of the second study was more modest (62 teachers), as this was an experiment, not a survey. 277
Besides the analysis of experimental data, a qualitative analysis of teacher diary entries will
follow in a further publication, to provide triangulation and thus to increase the trustworthiness
of the obtained results.
The theoretical framework of the current research was Kolb’s (1984) Experiential
Learning Theory and Schön’s (1983) Reflective Teaching Theory. According to Kolb’s theory,
both teachers and students learn by experience and then turn it into knowledge, while, according
to Schön, professional practice requires the ability to reflect over it. These ideas, in turn, were
based on John Dewey’s (1993) hands-on activities ideas.
The research design included the following stages:
• Developing the pre-experimental and post-experimental questionnaire and its
piloting (January 2020);
• Sample selection (February 2020);
• Holding the pre-experimental questionnaire (February 2020);
• Holding intervention in the experimental group and ordinary / traditional work in the
control group (9 weeks) (February-April 2020)
• Holding the post-experimental questionnaire (May 2020);
• Analysis of obtained results (May 2020).
Sample Selection
Teachers from across Georgia were shortly informed online (https://www.facebook.com/

groups/138750106331227/, https://www.facebook.com/ETAG-in-Georgia-152713621457305,
and https://www.facebook.com/search/top?q=Teachers%20of%20Georgia) about teacher
diaries, their goals, types, advantages, and challenges. They were shown some sample diary
entries. To make it more comprehensible for them, the information and the questionnaire were
in Georgian. All teachers who would volunteer, were invited to participate in the study. Besides,
some participants were recruited through the project ERASMUS+ (Assessment Tools for New
Learning Environments in Higher Education Institutions. 2017-2020, # ASSET 585587-EPP-
1-2017-1-IL-EPPKA2-CBIHE-JP) and the conference within the project entitled The First
Online Academic Writing Conference IFAW. The researcher also presented the current study
at various teachers’ meetings throughout Georgia during 2019/2020 educational year and
informed them that the questionnaire was available electronically or they could address the
researcher personally to get a copy.
After this they (also online) answered a pre-experimental self-assessment questionnaire,
according to which the level of their reflective thinking, communicative and professional skills
was defined.
Table 1
Participants’ Demographic Data
School / group Control group: totally 14 schools; 31 Experimental group: 15 schools, 31

teachers teachers
Private schools 1 school, 4 teachers 3 schools; 4 teachers
Public School 13 schools, 27 teachers 12 schools: 27 teachers
Tbilisi schools 8 schools; 21 teachers 11 schools: 21 teachers
Regional schools 7 schools from 5 regions 7 schools from 5 regions
Teacher gender M F M F
1 30 0 31
Teaching 1-5 years – 5; 6-10 years – 4; 11-20 – 13; 1-5 years – 5; 6-10 years – 6; 11-20 years –
experience above 20 - 9 15; above 20 - 5

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
278 The population of the study was 64,000 school teachers in Georgia, according to Geostat.
ge (2020)). The only way to reach them during the pandemic and lockdown was via social
media, and teachers were quite reluctant to be involved in any study. Although the total number
of volunteers (62) obviously does not allow to speak about a representative sample, they come
from totally 16 schools: 11 schools in the capital and seven schools from six (out of 12) regions
in Georgia, three private and 13 public schools. Teachers from all major types of schools and
geographic distribution were involved. The teaching experience of the teachers ranged from
1-5 years to over 20 years, and all groups according to experience were represented in both
groups quite similarly. Therefore, the application of cluster and stratificational approach lets the
researcher generalize the results for the Georgian context.
Procedure
Those 31 teachers who volunteered to conduct the diaries and did so for 9 weeks (three
weeks a dialogue diary with the trainer, three weeks a dialogue with a colleague at their choice,
and three weeks an individual diary) were the ‘experimental group’ (inverted commas are used
as they never came together as a group, but underwent the same treatment), while the rest of
the respondents (also 31 teachers) did teaching the way they have always done (the control
group). While the experimental group were writing dialogue diaries with the trainer, they better
understood how to write it. While writing dialogue diaries with colleagues, they continued to
improve their entries/analyses, benefitting from sharing with each other. Eventually, they were
able to write really good diary entries.
After that period the teachers had to answer a post-experimental questionnaire questions.
The questions for the control group were identical to the pre-experimental group questions,
while the experimental group teachers had to answer, besides those, several additional questions
concerning their experience of diary-writing.
Data Collection
Data resulted from a questionnaire were made up by the researcher. As already mentioned,
the questionnaire was mostly held online, as during the pandemic this was the most convenient
form as well as this was the fastest way to reach more participants. However, some (quite few)
participants preferred to hand the questionnaires in personally. The dialogue diaries were held
online, individual diaries were mostly held online. Some teachers preferred to hold them on-
paper and then scan for the researcher.
The majority of items had to be assessed by the respondents in a 5-point Likert scale),
while one item in the post-experimental questionnaire for the experimental group was open-
ended, which enabled the respondents to express their opinion of the treatment they had
undergone.
The tool was developed based on several existing inventories: the reflective thinking
skills part on Karpov (2003) and Mirzaei et al. (2913), while the communicative and professional
skills part based on Good and Mulryan (1990). However, as the questions were modified and
applied partially and were translated into Georgian, the native language of the respondents, for
providing the face, construct, and content validity of the questionnaire it was assessed by three
experts in the field. Their recommendations were taken into consideration (some items were
added, omitted, or reformulated). Also, in order to ensure the reliability of the questionnaire
items and the questionnaire on the whole, the test-retest approach was applied to provide the
reliability of the test. The results are shown in Table A1 in the Appendix. The answers to the
items marked by asterisks (*) were calculated in reverse order (answer “1” as “”5, “2” as “4”,
“3” as “3”, “4” as “2”, and “5” as “1”), as they are negative by nature. This remark concerns

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
the following tables as well. To formulate some items positively while others negatively was 279
necessary to provide that the respondents assess the items attentively, not mechanically (such
as all “5-”s) (Croasmun & Ostrom, 2011). They were warned about it. For statistics reversed
calculation was applied, as the researcher needed to ‘average mean results’ in order to see the
treatment effect on the whole. While assessing the obtained mean results, the results below 2.67
were viewed as negative, from 2.67 to 3.33 as marginal, and 3.34 and above as positive (Joshi
et al, 2015; Lipsey et al., 2012).
It is seen from Table A1 in the Appendix, most answers in the ‘retest’ are the same
(see part 1 items 2, 5, 9, part 2 items 3 and 10, part 3 items 1 and 7) or almost the same (the
remaining items). The correlation of the two measurements was found to be very high (1.00 or,
more exactly, .997), and the significance p=.000<.001, which means that the questionnaire was
reliable.
Research Results
The obtained data were first entered to Excel sheets and then statistically treated with
SPSS 26 software. For brevity, the items are not repeated in the following tables, but they are
the same as above. The results (rounded to 100th) are presented in Tables 2-5 below. The ‘overall
mean’ in the table stands for the mean result for all items in the part, while the ‘total mean’
stands for the mean result of the whole questionnaire.
Table 2
Experimental Group Pre-Experimental Results
Items M Mdn Mode SD Skewness Kurtosis

Part 1: Reflective thinking skills
1 4.39 4.00 4.00 .56 -.14 -.85
2* 4.16 4.00 4.00 .90 -1.52 3.76
3 4.23 4.00 4.00 .67 -1.01 2.87
4 4.10 4.00 4.00 .75 -0.16 -1.12
5 4.13 4.00 4.00 .72 -.78 1.35
6 4.35 4.00 4.00 .61 -.35 -.57
7 4.26 4.00 5.00 .89 -1.15 .78
8 4.61 5.00 5.00 .50 -.49 -1.89
9 4.39 4.00 4.00 .62 -.46 -.56
10 4.00 4.00 4.00 1.13 -1.20 .69
11* 4.06 4.00 4.00 .63 -.04 -.26
12* 4.00 4.00 4.00 .78 .00 -1.29
13 4.32 4.00 4.00 .65 -.44 -.61
Part 1 overall mean 4.25
Part 2: Communicative skills
1 3.90 4.00 4.00 .94 -.82 1.30
2* 3.19 3.00 4.00 1.22 -.40 -.64
3 4.26 4.00 4.00 .51 .35 -.23
4* 3.74 4.00 4.00 1.29 -.98 -.04
5* 3.61 4.00 4.00 1.26 -.81 -.21

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
280 6 3.39 4.00 4.00 .50 .49 -1.89

7 4.32 4.00 4.00 .83 -2.17 7.55
8 3.77 4.00 4.00 1.15 -1.23 1.34
9* 4.48 5.00 5.00 .85 -1.85 3.10
10 3.55 4.00 4.00 1.09 -.46 -.48
Part 3: Professional skills
1 3.93 4.00 4.00 .73 -2.12 8.56
2 4.19 4.00 4.00 .75 -.85 1.06
3 4.16 4.00 4.00 .69 -.22 -.77
4 4.10 4.00 4.00 .54 .10 .70
5 4.32 4.00 4.00 .70 -1.17 2.56
6 4.35 4.00 4.00 .61 -.35 -.57
7 4.13 4.00 4.00 .85 -.96 .83
8 4.16 4.00 4.00 .73 -.81 1.18
9* 4.55 5.00 5.00 .57 -.78 -.36
10* 4.52 5.00 5.00 .81 -2.06 4.38
11* 4.55 5.00 5.00 .85 -2.08 3.82
12* 4.65 5.00 5.00 .55 -1.27 .76
13* 4.68 5.00 5.00 .54 -1.46 1.36
14* 3.84 4.00 4.00 1.07 -.71 -.66
15* 4.19 4.00 5.00 .91 -.98 .26
Total mean 4.15
The mean, median and mode were close to each other, which makes the obtained means
quite trustworthy. The majority of the mean results in the experimental group were close
(3.39<M<3.90), equal to or more than 4, which reveals that on the whole the respondents quite
positively assessed their reflective thinking, communicative and professional skills. Only one
mean result was low (M=3.19). The standard deviations ranged from average (SD=0.50) to high
(SD=1.29). This means that the respondents’ opinions on many items were not homogeneous.
Skewness was mostly negative and above -3. This reveals that there were more results
that are lower than the mean result. Therefore, in reality many results were to a certain degree
lower than the mean. There were some items, whose kurtosis was more than +1. This means that
the distribution of assessments to these items was too high and the distribution is not normal.
These were item 1 and 2 in part one, items 1, 8 and especially 7, in part 2 they were items 1,
7, and 9, while in part 3 these were items 2, 8, 13, and especially 1 and 10. The items whose
kurtosis was less than -1 reveal more unity of opinion. These were items 4, 8, and 1 in part 1,
item 6 for part 2, and there were no such items for part 3. The experimental group teachers
assessed their professional skills the highest (M=4.29), then reflective thinking (M=4.25) and
the lowest, still positive enough their communicative skills (M=3.82). The total mean equaled
4.15, which is quite positive.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Table 3 281
Control Group Pre-Experimental Results
M Mdn Mode SD Skewness Kurtosis

1 4.29 4.00 4.00 0.64 -0.59 -0.34
2* 3.84 4.00 4.00 0.97 1.43 -1.06
3 4.26 4.00 4.00 0.51 -0.23 0.35
4 4.10 4.00 4.00 0.83 -1.53 -0.19
5 4.45 4.00 4.00 0.51 -2.09 0.20
6 4.30 4.00 4.00 0.66 -1.24 0.92
7 3.42 4.00 4.00 1.06 1.06 -1.13
8 4.65 5.00 5.00 0.49 -1.71 -0.64
9 4.42 4.00 5.00 0.62 -0.51 -0.57
10 3.71 4.00 4.00 1.13 -0.47 -0.56
11* 4.10 4.00 4.00 0.79 1.65 -1.06
12* 4.06 4.00 4.00 0.95 .331 -0.94
13 4.29 4.00 4.00 0.46 -1.13 0.97
1 4.06 4.00 4.00 1.73 -1.01 -0.10
2* 3.68 4.00 3.00 0.94 -0.89 -0.04
3 4.39 4.00 4.00 0.50 -1.89 0.49
4* 4.26 4.00 4.00 0.73 -0.94 -0.45
5* 3.97 4.00 4.00 1.02 -0.42 -0.75
6 4.42 4.00 4.00 0.50 -2.02 0.34
7 4.29 4.00 4.00 0.64 3.95 -1.15
8 4.19 5.00 5.00 1.08 -0.13 -1.10
9* 4.19 5.00 5.00 1.08 -0.13 -1.10
10 3.61 4.00 4.00 1.20 -0.95 -0.41
1 4.16 4.00 4.00 0.58 0.00 -0.01
2 4.29 4.00 4.00 0.46 -1.13 0.97
3 4.19 4.00 4.00 0.79 1.07 -2.09
4 4.23 4.00 4.00 0.56 -0.103 0.50
5 4.35 5.00 5.00 0.80 1.08 -1.17
6 4.39 4.00 5.00 0.66 -0.55 -0.64
7 4.10 4.00 4.00 0.91 3.93 -1.63
8 3.97 4.500 5.00 0.92 1.96 -1.53
9* 4.30 5.00 5.00 0.57 -0.78 -0.36
10* 4.43 5.00 5.00 0.86 1.02 -2.41
11* 4.35 4.00 5.00 0.75 1.73 -1.21

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
282 12* 4.52 5.00 5.00 0.68 1.81 -1.79

13* 4.58 5.00 5.00 0.67 1.89 -2.07
14* 3.90 4.00 4.00 1.16 0.06 -1.02
15* 4.32 4.00 5.00 0.70 -1.75 -0.55
Total mean 4.09
quite trustworthy. Similarly to the experimental group, the majority of the mean results in the
control group were close to (3.61<M<3.97), equal or more than 4, which reveals that on the
whole the respondents quite positively assessed their reflective thinking, communicative and
professional skills. One result was quite low (M=3.42) however, this result can also be viewed
as positive. The standard deviations ranged from average (SD=0.50) to high (SD=1.73). This
means that the respondents’ views were not too homogeneous. The skewness was mostly
negative and above -3. This reveals that there were more results that were lower than the mean
result. Therefore, in reality many of their results were to a certain degree lower than the mean.
There was a perspective to improve them. There were no items whose kurtosis was more than
+1. This reveals that the assessments were more or less homogeneous. The kurtosis of items 2
and 7 in part 1, items 7, 8, and 9 in part 2, and of items 3, 5, 7, 8, 10, 11, 12, 13, and 14 were less
than -1. This means that the opinions of different respondents on these items were rather similar
to each other. The control group teachers assessed their professional skills the highest (M=4.31),
then reflective thinking (M=4.15) and the lowest, but still positively their communicative skills
(M=4.10). The total mean equaled 4.09, which is quite positive.
The overall means of both groups were close enough to each other, so the groups were
quite comparable.
Table 4
Experimental Group Post-Experimental Results

1 4. 42 5.00 5.00 1.12 4.74 -2.30
2* 4. 26 4.00 4.00 0.73 1.69 -1.00
3 4.58 5.00 5.00 0.50 -2.02 -0.34
4 4.58 5.00 5.00 0.56 -0.08 -0.93
5 4.58 5.00 5.00 0.76 3.60 -1.96
6 4.48 5.00 5.00 0.89 7.01 -2.38
7 4.35 4.00 4.00 0.66 -0.60 -0.53
8 4.77 5.00 5.00 0.43 -0.11 -1.38
9 4.48 5.00 5.00 0.68 -0.15 -0.97
10 4.00 4.00 5.00 1.00 -.637 -.641
11* 4.06 4.00 4.00 0.63 -0.04 -0.26
12* 4.00 4.00 4.00 0.78 0.00 -1.29
13 4.48 5.00 5.00 0.93 7.03 -2.51

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
1 4.03 4.00 4.00 0.91 0.40 -0.91 283
2* 3.87 4.00 4.00 0.99 -0.52 -0.60

3 4.58 5.00 5.00 0.56 -.077 -.933
4* 4.06 5.00 5.00 1.31 0.96 -1.44
5* 3.68 4.00 5.00 1.33 -0.95 -0.55
6 4.65 5.00 5.00 0.49 -1.71 -0.64
7 4.58 5.00 5.00 0.56 -0.08 -0.93
8 3.97 4.00 4.00 0.85 -0.72 -0.30
9* 4.58 5.00 5.00 0.56 -.077 -.933
10 4.00 4.00 4.00 0.96 0.16 -0.68
1. 4.32 4.00 5.00 0.79 1.00 -1.10
2 4.52 5.00 5.00 0.57 -0.57 -0.64
3 4.52 5.00 5.00 0.63 -0.05 -0.94
4 4.35 4.00 4.00 0.49 -1.71 0.64
5 4.61 5.00 5.00 0.72 5.20 -2.18
6 4.61 5.00 5.00 0.50 -1.89 -0.49
7 4.45 5.00 5.00 0.62 -.40 -0.69
8 4.10 4.00 4.00 0.91 3.07 -1.34
9* 4.45 5.00 5.00 0.77 2.19 -1.48
10* 4.60 5.00 5.00 0.56 0.18 -1.04
11* 4.43 5.00 5.00 0.86 2.64 -1.71
12* 4.63 5.00 5.00 0.49 -1.78 -0.58
13* 4.74 5.00 5.00 0.44 -.70 -1.16
14* 4.23 4.00 4.00a 0.80 .0.40 -0.86
15* 4.16 4.00 4.00 0.82 1.41 -1.09
Total mean 4.36
Part 4: Assessment of experiment results
1 According to my experience, conducting a reflective 4.52 5.00 5.00 0.51 -2.14 -0.07
diary is useful.
As result of conducting a reflective diary, my following
skills improved:
2. 1. reflective thinking 4.58 5.00 5.00 0.50 -2.02 -0.34
2. 2. critical thinking 4.55 5.00 5.00 0.51 -2.10 -0.20
2.3. communication 4.39 4.00 5.00 0.67 -0.55 -0.64
2.4. self-assessment 4.70 5.00 5.00 0.47 -1.24 -0.92
2.5. professional writing 4.42 5.00 5.00 0.72 2.82 -1.42
2.6. awareness of my teaching style 4.58 5.00 5.00 0.56 -.077 -0.93
2.7. developing interesting ideas 4.50 4.50 4.00a 0.51 -2.15 0.00
2.8. professional skills 4.48 5.00 5.00 0.57 -0.72 -0.51
2.9. co-operation with colleagues 4.35 4.00 4.00 0.61 -0.57 -0.35

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
284 The mean, median and mode were close to each other, which makes the obtained means
quite trustworthy. The majority of the mean results in the experimental group were equal to
or more than 4, only 3 results (M=3.68, M=3.87 and M=3.97) were close to 4 (these results
are also positive). This reveals that on the whole the respondents quite positively assessed
their reflective thinking, communicative and professional skills. Their mean level increased
compared to pre-experimental results (see Table 6).
The answers to the additional part assessing the teachers’ views on the treatment applied
were also all positive (4.39<M<4.70). Therefore, the teachers were satisfied by applying teacher
reflective diaries. The skewness was mostly negative and above -3. This reveals that there were
more results that were lower than the mean result. Therefore, in reality many of their results
were to a certain degree lower than the mean.
The standard deviations were still from average (SD=0.43) to high (SD=1.33). This means
that the respondents’ views were still not too homogeneous, however, the standard deviations
decreased a little, so there was more unity of opinion compared to the pre-experimental results.
There were no items whose kurtosis was more than +1. This reveals that the assessments were
more or less homogeneous. The kurtosis of items 1, 5, 6, 12 and 13 in part 1, item 4 in part
2, and of items 1, 5, 8, 9, 10, 11, 13 and 15 were less than -1. This means that the opinions of
different respondents on these items were rather similar to each other (the distribution was close
to normal).
On the whole, the post-experimental results of the experimental group were better than
the pre-experimental ones. The overall means of all parts were above 4. The total mean increased
from 4.15 to 4.36. All means of the part 4 (additional questions which cannot be compared to
pre-experimental results) were above 4 (4.35-4.58, the overall mean was 4.51), which is a very
good achievement.
Table 5
Control Group Post-Experimental Results

Part 1. Reflective thinking skills
1 3.55 4.00 3.00 0.57 -0.81 0.38
2* 3.74 4.00 4.00 0.82 -0.21 -0.27
3 3.74 4.00 4.00 0.82 -0.21 -0.27
4 4.06 4.00 4.00 0.73 -1.01 -0.10
5 4.39 4.00 4.00 0.62 -0.56 -0.46
6 4.45 5.00 5.00 0.72 -0.38 -0.95
7 3.42 3.00 4.00 1.09 -1.26 0.06
8 4.68 5.00 5.00 0.48 -1.46 -0.80
9 3.71 4.00 4.00 0.82 0.08 -0.54
10 3.77 4.00 4.00 0.76 1.26 -1.03
11* 4.10 4.00 4.00 0.79 1.65 -1.06
12* 4.06 4.00 4.00 0.95 0.33 -0.94
13 3.55 4.00 4.00 0.81 -0.28 -.017
Part 2. Communicative skills
1 4.06 4.00 4.00 0.81 1.05 -.0.92
2* 3.55 4.00 4.00 1.06 -1.13 -0.23

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
3 4.52 4.00 5.00 0.51 -2.14 -0.07 285
4* 3.93 4.50 5.00 1.39 -.080 -1.12

5* 3.68 4.00 5.00 1.25 -0.53 -0.65
6 4.32 4.00 5.00 0.70 -0.75 -0.56
7 4.26 5.00 5.00 1.12 3.27 -1.90
8 3.23 3.00 2.00 1.18 -1.19 0.06
9* 3.87 4.00 4.00 0.72 0.41 -0.38
10 3.32 4.00 4.00 1.11 0.43 -1.01
1 3.39 3.00 4.00 0.67 -0.55 -0.64
2 3.71 4.00 4.00 0.78 -0.01 -0.31
3 3.52 4.00 4.00 0.72 -0.04 -1.19
4 4.27 4.00 4.00 0.58 -0.36 -0.09
5 4.42 4.00 4.00 0.50 -2.02 0.34
6 3.81 4.00 4.00 0.75 -0.14 -0.17
7 3.71 4.00 4.00 0.64 0.62 -0.46
8 3.65 4.00 4.00 1.02 1.50 -1.23
9* 3.65 4.00 4.00 0.66 0.90 -0.95
10* 4.48 4.00 5.00 0.57 -0.72 -0.51
11* 4.35 4.00 4.00 0.71 2.59 -1.24
12* 4.29 4.00 4.00 0.90 5.82 -2.09
13* 4.52 4.00 5.00 0.68 4.81 -1.79
14* 3.68 4.00 4.00 0.75 0.76 -0.91
15* 4.13 4.00 4.00 0.81 1.51 -1.06
Part 3 overall results 3.97
Total mean results 4.00
quite trustworthy. About half of the mean results in the control group were equal to or more
than 4, the majority of the results were positive (M>3.34), only 2 results (M=3.23 and M=3.32)
can be assessed as marginal (neither clearly positive, nor clearly negative), which reveals that
on the whole the respondents quite positively assessed their reflective thinking, communicative
and professional skills. The mean level of communicative slightly decreased (4.103.87), of
professional skills significantly decreased (4.313.97), while reflective thinking skills slightly
increased (4.154.21) (see Table 6). The skewness was mostly negative and above -3, which
reveals that there were more results that were lower than the mean result, therefore, in reality
many students’ results were to a certain degree lower than the mean, and there was a perspective
to improve them.
The standard deviations still ranged from average (SD=0.50) to high (SD=1.39), which
means that the respondents were not too homogeneous in their assessments. There were no
items whose kurtosis was more than +1, which reveals that the assessments were more or less
homogeneous. The kurtosis of items 10 and 11 in part 1, items 4, 7, and 10 in part 2, and of
items 3, 8, 11, 12, 13 and 15 were less than -1, which means that the opinions of different
respondents on these items were rather similar to each other.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
286 On the whole, the post-experimental results of the control group were lower than the pre-
experimental ones. The total mean decreased from M=4.31 to M=3.96. This may be explained
by the pandemic and the stress of unexpectedly moving to totally online teaching, but, to name
the cause, additional research is needed.
In table 6 below, pre-experimental and post-experimental mean results of the experimental
and control groups are compared to each other in order to see whether the groups decreased,
maintained or increased the level of reflective thinking, communicative and professional skills
after the experiment.
Table 6
Comparison of the Experimental and Control Group Pre- and Post-Experimental Results
Experimental group Control group
Pre-experimental Post-experimental Pre-experimental Post-experimental

results (Mmax =5) results (Mmax =5) results (Mmax =5) results (Mmax =5)
Reflective thinking
4.25 4.39 4.15 4.21
skills
Communicative skills 3.82 4.20 4.10 3.87
Professional skills 4.29 4.41 4.31 3.97
The table reveals that, as assessed by the participants according to a 5-point Likert
scale, in the experimental group all three skills increased as result of treatment, while in the
control group reflective thinking skills slightly increased, communicative skills experienced a
minor decrease and professional skills a notable one. Therefore, the experiment results were
successful for the experimental group and not successful for the control group. To see whether
the improvements in the experimental group were statistically significant, and whether the
changes in the control group were statistically significant, a t-test was conducted (Tables 7 and
8).
Table 7
T-Test for Experimental Group Results
One-Sample Statistics
N M SD SEM
Variable 1 3 4.12 .26 .15
Variable 2 3 4.33 .12 .07
One-Sample test
Test value = 0
Mean 95% CI of the difference
T df Sig. (2-tailed)
Difference Lower Upper
Variable 1 27.39 2 .000 4.12 3.47 4.77
Variable 2 64.76 2 .000 4.33 4.05 4.62

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Therefore, at 95% confidence interval of the difference which is normally applied in 287
educational research (Cohen et al, 2018), the significance p=.0001<.001. This result implied that
the difference between the pre-experimental and post-experimental results in the experimental
group is statistically significant. Thus, the positive hypotheses (H1a, H1b, and H1c) were confirmed.
Table 8
T-Test for Control Group Results
One-Sample Statistics
N M SD SEM
Variable 1 3 4.19 .11 .06
Variable 2 3 4.02 .17 .10
One-Sample test
Test value = 0
Mean 95% CI of the difference

T df Sig. (2-tailed)
Difference Lower Upper
Variable 1 66.11 2 .00 4.18667 3.9142 4.4592
Variable 2 38.81 2 .00 4.01667 3.5826 4.4507
Therefore, at 95% confidence interval of the difference which is normally applied in

educational research (Cohen et al, 2018), the significance p=0.0001<0.05. This result implied
that the difference between the pre-experimental and post-experimental results in the control
group is statistically significant.
To find out to what degree was the impact of diary application in the experimental
group positive, Person correlation was assessed.
Table 9a
Pearson Correlation between Reflective Thinking Skills and Diary Writing
Pre-experimental Post-experimental
r 1 0.73
Reflective thinking skills Significance (2-tailed) 0.05
N 13 13
r .73 1
Significance (2-tailed) 0.005
N 13 13

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
288 Table 9b
Pearson Correlation between Communicative Skills and Diary Writing
r 1 0.53
Communicative skills Significance (2-tailed) 0.12
N 10 10
r 0.53 1
N 10 10
Table 9c
Pearson Correlation between Professional Skills and Diary Writing
r 1 0.474
Professional skills Significance (2-tailed) 0.74
N 15 15
r 0.474 1
N 15 15
From the Tables 9a, 9b, and 9c it can be concluded that the correlation is positive and
average, the highest for reflective thinking skills (r =0.73), followed by communicative skills
(r= 0.53) and professional skills (r=0.47).
As for the comments that the teachers made in the open-ended question, they mention the
following advantages of diary writing:
• it enables to follow the progress of each individual student;
• it has been especially useful during online teaching as anyway we had to collect
information about every student;
• it is like additional memory – to keep in mind all achievements, problems, activities,
etc.;
• lessons are ephemeral – it’s over and that’s all, a diary helps to keep it in memory
for analysis;
• it is a good tool for self-assessment and self-efficiency; one learns to assess oneself
more objectively and thoroughly;
• provides systematic self-observation;
• for teachers who enjoy personal diary writing, it’s also enjoyable;
• it motivates the teacher to work better;
• it develops analytic and critical skills;
• it enables the teacher to see her strong and weak points;
• it helps to visualize problems;
• it helps to clarify one’s teaching style and strategies;
• it makes planning and teaching more thoughtful;
• it’s like a mirror for a teacher;
• the peer dialogue helps to obtain colleagues’ feedback;

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
• it promotes professional development, a deeper understanding of theoretical 289

knowledge and its link with teaching practice;
• it develops academic writing quality;
• it improves the quality of teaching.
Six respondents out of 31 mentioned that they would continue diary writing after the
research would be over, and one wrote she would write it from time to time. One respondent
expressed the idea that for novice teachers writing a diary should be compulsory.
As for challenges connected with diary writing, a couple of teachers mentioned it was
time-consuming, and they had to find special time for it. One teacher mentioned she was afraid
in the beginning she would not be able to do it, but after a couple of entries it became easy.
Discussion
Although there are many publications concerning the role of teacher diaries, most of
them deal with other than skill development benefits such as enhancing social work (Vinjamuri
et al. 2017), teaching English writing (Farrah, 2012), students’ self-regulated learning strategies
(Al-Rawahi & Al-Balushi, 2015), etc. Besides, the analysis of the existing studies dealing
with reflective diary writing brought the researcher to the conclusion that the majority of the
articles involved teacher students, and there was a lack of research dealing with actual teachers.
Besides, the available publications are qualitative (interview, observation, case studies) and
non-experimental qualitative (survey) ones. The current study tried to at least partially fill the
existing gap.
The current study revealed that the participant teachers had a positive attitude towards
reflective diary writing. It was also found that there was a positive relationship between diary
writing and the development of reflective thinking, communicative and professional skills.
Göker (2016) held a research with 16 teacher certification students at a Turkish university
who applied a reflective diary during their 6-month-long teaching practice. He applied surveys
and interviews. 91% of the participants expressed a positive opinion of diary writing. Besides,
they described advantages and challenges of diary writing, which also were in line with the
current research findings. In the current research all items in the pre-experimental questionnaire
were assessed positively by the control group, while only one item was assessed on a marginal
level (neither positive, nor negative: M=3.19). In the post-experimental questionnaire, all
teachers in the experimental group assessed the application of teacher diary positively, while
in the control group only two items were assessed as marginal (M=3.23 and M=3.32). This
comparison reveals that the current results were in line with Göker’s (2016) results.
In Insuasty and Castillo (2010) study six teacher students were applying a reflection
diary during their six-month practicum in a Colombian university. They held a curriculum
action research. “Observational and non-observational techniques were used to collect data.
The former were journals, blog group discussions and conferences. The latter were metaphors
and questionnaires” (ibid, p. 92). The results revealed that they enriched their understanding
of reflective teaching and developed their reflective thinking skills. While describing a
teacher before this experience, they mentioned such teacher responsibilities as controlling the
students’ progress, applying innovations, but none of them mentioned that teachers should
be reflective practitioners. After the research was completed, they realized the importance of
reflective teaching. In the current research, the teachers from the experimental group, while
writing the diaries, had to assess their own and their colleague’s practice, which stimulated
the development of their reflective skills. If before the experiment they already assessed their
reflective skills quite positively (M=4.25), after the experiment the level of their skills increased
(M=4.39). Therefore, Insuasty and Castillo (2010) findings are also in compliance with the
current research.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
290 Bashan and Holsblat (2017) held a qualitative study, analyzing 36 Israeli teacher students’
entries in reflective diaries (three years, 12 students a year). The research revealed that reflective
diaries supported teacher students’ communication with each other, school teachers and their
instructor, helped to build mutual trust. Keeping the diaries improved their lesson planning
skills. Sharing knowledge and experiences was also emphasized by both practicing teachers and
student teachers. In the current research, the teachers communicated with the trainer for the first
three weeks, then with their peers for the next three weeks, which significantly increased their
communication skills (before the experiment their level was M=3.82, while post-experimental
level became M=4.20). The above findings are also in congruence with the current research.
Moghaddam et al. (2020) conducted a mixed-methodology study: a questionnaire
administered to 250 Iranian English teachers and an experiment with 60 teachers, in which
the experimental group was conducting reflective diaries. The diaries were found to develop
cognitive, metacognitive, affective, critical, and practical skills. The teaching (i.e., professional)
skills of the experimental group teaching substantially increased. Almusharraf (2020) involved
eight Saudi female teachers undergoing an 11-week teaching practicum. They confessed that
reflective practice let them develop their own teaching philosophy. In the current study, the
teachers had to purposefully use the professional terminology more thoughtfully as well as to
apply their theoretical knowledge in practice. This enabled them to significantly increase their
professional skills (before the experiment their level was M=4.29, while post-experimental
level became M=4.41). The findings by Moghaddam et al. (2020) and Almusharraf (2020) are
consistent with the current study findings.
On the whole, the current research found the positive impact of conducting a teacher diary
on three skills (reflective, communicative and professional), while the other available research
studies deal with one or two benefits of reflective diaries. Besides, it was an experimental
research (compared to descriptive research in all studies except Moghaddam et al), which
makes its results more scientifically valuable. And, finally, this is the first research in Georgia
on the given issue.
Research Limitations and Further Research Directions
The number of participants of the research was limited, which does not make its findings
generalizable, so it would be useful to replicate the research, both in Georgia and other countries.
It would also be useful to investigate the impact of teacher reflective journals on students’
achievement.
Conclusions
The conducted research enriched the scarce empirical data on the issue of teacher diary
impact on the improvement of their reflective, communicative, and professional/pedagogical
skills. It is one of the very few research studies (Moghaddam et al., 2020; Almusharraf, 2020)
of a complex nature (covering several skills) held with teachers and not teacher students as
participants. It has shown that a reflective diary benefits teachers as well.
The research gave a positive answer to the research questions (whether writing a
reflective diary developed teachers’ reflective thinking, communicative and professional skills).
The impact was found through an experiment and accompanying questionnaire to be positive
and average. The strongest impact was shown on reflective thinking skills (=0.73), while the
impact on communicative (0.53) and professional (0.47) skills was less. The experimental group
increased these skills significantly, while in the control group, without any special treatment,
reflective thinking skills slightly increased, however, communicative and professional skills
decreased.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
According to the study results, the application of teacher reflective diary can be 291
recommended as an effective tool for the development of reflective thinking, communication, and
professional qualification. The quality of teaching will increase if teachers, at least periodically,
apply reflective diary writing. This, in turn, will help to improve students’ knowledge and skills
and thus help solve the problem stated in the research.
References
Aharonian, N. (2016). I thought to myself: What a long way I’ve come …: Teacher writing for
professional learning. An International Journal of Teachers’ Professional Development, 20(2),
213-228. https://doi.org/10.1080/13664530.2016.1138998
Ahmed, A. M. (2019). Students’ reflective journaling: An impactful strategy that informs instructional
practices in an EFL writing university context in Qatar. Reflective Practice, 20(4), 483-500.
https://doi.org/10.1080/14623943.2019.1638246
Almusharraf, A. M. (2020). Student teachers’ development of reflective practice concerning
teaching philosophy and peer observations. Arab World English Journal, 11(4) 547-564.
https://doi.org/10.24093/awej/vol11no4.35
Al-Rawahi, N., & Al-Balushi, S. (2015). The effect of reflective science journal writing on students’ self-
regulated learning strategies. International Journal of Environmental & Science Education, 10(3),
367-379. https://files.eric.ed.gov/fulltext/EJ1069260.pdf
Antoniou, P., Kyriakides, L., & Creemers, B. P. M. (2015). The dynamic integrated approach
to teacher professional development: Rationale and main characteristics. An International Journal of
Teachers’ Professional Development, 19(4), 535-552. https://doi.org/10.1080/13664530.2015.1079550
Ba s h a n , B . , & Hol sbl a t , R. (2017). Re fl e ctive journals as a res earch tool: The cas e
o f s tu d e nt t e a c he rs’ de ve l opm e nt o f teamw ork. Cogent Education, 4, 1-15.
h ttp s ://d oi .org/ 10.1080/ 2331186X.2017.1374234
Chien, C. W. (2018). Analysis of six Taiwanese EFL student teachers’ professional learning from writing,
discussing, and analyzing critical incidents. An International Journal of Teachers’ Professional
Development, 22(3), 339-354. https://doi.org/10.1080/13664530.2017.1403368
Chong, S., & Cheah, S. M. (2009). A values, skills, and knowledge framework for initial teacher
preparation programmes. Australian Journal of Teacher Education, 34(3), 1-17.
Cohen, L., Manion, L., & Morrison, K. (Eds.). (2018). Research methods in education. 8th ed. Routledge.
Croasmun, J., & Ostrom, L. (2011). Using Likert-type scales in the social sciences. Journal of Adult
Education, 40(1), 19-22.
Dewey, J. (1993). How we think. A restatement of the relation of reflective thinking to the educative
process (revised edition). Heath.
Dincel, B., & Savur, H. (2018). Diary keeping in writing education. Journal of Education and Training
Studies, 7(1). https://doi.org//48-59.10.11114/jets.v7i1.3758
Di Pardo Léon-Henri, D. (2021). 14 reasons teachers should keep a reflective journal. Reflective Teaching
Journal. https://reflectiveteachingjournal.com/reasons-teachers-should-keep-a-reflective-journal
Farrah, (2012). Reflective journal writing as an effective technique in the writing process. An- Najah
University Journal of Research (Humanities), 26(4), 998-1025.
Finlay, L. (2008). Reflecting on reflective practice. The Open University. Practice-Based Professional
Center. http://ncsce.net/wp-content/uploads/2016/10/Finlay-2008-Reflecting-on-reflective-
practice-PBPL-paper-52.pdf
Gogoladze, N. (2019). Refleqsia da sastsavlo protsesi [in Georgian: Reflection and educational process].
https://en.calameo.com/read/0044967375af0ea5e84c8
Göker, S. D. (2016). Use of reflective journals in development of teachers' leadership and teaching skills.
Universal Journal of Educational Research, 4(12A), 63-70.
Good, T. L., & Mulryan, C. (1990). Teacher ratings: A call for teacher control and self-evaluation. In J.
Millman & L. Darling-Hammond (Eds.). The new handbook of teacher evaluation: Assessing:
elementary and secondary school teachers (pp. 191- 215). SAGE.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
292 Gore, J., Lloyd, A., Smith, M., Bowe, J., Ellis, H., & Lubans, D. (2017). Effects of professional
development on the quality of teaching: Results from a randomised controlled trial of Quality
Teaching Rounds. Teaching and Teacher Education, 68, 99-113.
Hilton, G., Assunção Flores, M., & Niklasson, L. (2013). Teacher quality, professionalism, and
professional development: Findings from a European project. An International Journal of
Teachers’ Professional Development, 17(4), 431-447.
Insuasty, E. A., & Zambrano Castillo, L. C. (2010). Exploring reflective teaching through informed
journal keeping and blog group discussion in the teaching practicum. Profile, 12(2), 87-105.
Joshi, A., Kale, S., Chandel, S., & Pal, D. K. (2015). Likert scale: Explored and explained. British Journal
of Applied Science & Technology, 7(4), 396-403.
Karpov, A. (2003). Refleksivnost kak psikhicheskoye svoistvo i metodika ee diagnostiki [Reflexivity as
psychological characteristic and methods of its diagnostic]. Psykhologicheskiy zhurnal, 24(5), 45-57.
Klimova, B. (2015). Diary writing as a tool for students’ self-reflection and teacher's feedback in the
course of academic writing. Procedia - Social and Behavioral Sciences, 197, 549 – 553.
Kolb, D. A. (1984). Experiential learning experience as a source of learning and development. Prentice
Hall.
Lipsey, Puzio, K., Yun, C., Hebert, M., Steinka-Fry, K., Cole, M., Roberts, M., Anthony, K., & Busick,
M. (2012). Translating the statistical representation of the effects of education interventions into
more readily interpretable forms. National Center for Special Education Research, Institute of
Education Sciences, U.S. Department of Education.
Liu, L., & Zhang, Y. (2014). Enhancing teachers' professional development through
reflective teaching. Theory and Practice in Language Studies, 4(11), 2396-2401.
https://doi.org//10.4304/tpls.4.11.2396-2401
Lopera Medina, S. (2013). Diary insights of an EFL reading teacher. Profile: Issues in Teachers’
Professional Development, 15(2), 115-126.
Mirzaei, F., Phang, F. A., & Kashefi, H. (2014). Measuring teachers’ reflective thinking skills. Procedia -
Social and Behavioral Sciences, 141, 640 – 647.
Moghaddam, R. G., Davoudi, M., Adel, S. M. R. et al. (2020). Reflective teaching through journal
writing: A study on EFL teachers’ reflection-for-action, reflection-in-action, and reflection-on-
action. English Teaching & Learning, 44, 277–296.
OECD (2018). PISA 2018 results. https://www.oecd.org/pisa/publications/pisa-2018-results.htm
Oelofsen, N. (2012). Developing reflective practice: A guide for students and practitioners of health and
social care. Lantern Publishing.
Patterson, D. (2014). Using writing journals as a means of increasing EFL writing fluency. Proceedings of
ICLC 2013: The 5th International Conference on Language & Communication, 81-91. December
12-13. Bangkok, Thailand.
Schön, D. (1983). The reflective practitioner: How professionals think in action. Temple Smith.
Spalding, E., & Wilson, A. (2002). Demystifying reflection: A study of pedagogical strategies that
encourage reflective journal writing. Teachers College Record, 104(7), 1393–1421.
Stevens, D. D., & Cooper, J. E. (2009). Journal keeping: How to use reflective writing for learning,
teaching, professional insight, and positive change. Stylus Publishing, LLC.
Taqi, H., Akbar, R., Al-Nouh, N., & Dashti, A. (2015). The effect of diary writing on EFL students’
writing and language abilities. British Journal of Education, 3(2), 75-91.
Tkavashvili, E. (2021). Awareness of the significance of reflective diaries for teachers’ professional
development. International Journal of Sciences: Basic and Applied Research, 55(3). In print.
Tsingos-Lucas, C., Bosnic-Anticevich, S., Schneider, C., & Smith, L. (2017). Using reflective writing
as a predictor of academic success in different assessment formats. American Journal of
Pharmaceutical Education, 81(1), article 8. https://doi.org/10.5688/ajpe8118
Ukrop, M., Švábenský, V., & Nehyba, J. (2019). Reflective diary for professional development of novice
teachers. In Proceedings of the 50th ACM technical symposium on computer science education
(SIGCSE ’19). February 27-March 2. Minneapolis, MN, USA. ACM, New York, NY, USA.
https://doi.org/10.1145/3287324.3287448
Vinjamuri, M., Warde, B., & Kolb, P. (2017). The reflective diary: An experiential tool for
enhancing social work students’ research learning. Social Work Education, 36(8), 933-945.
https://doi.org/10.1080/02615479.2017.1362379

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Appendix 293
Table A1. Defining the reliability of the questionnaire items
Items Variable 1: Variable 2:

Means (first Means (second
measurement) measurement)
1. When I fail to do something well at the lesson, first of all, I look 4.25 4.27
for a cause in myself for what I have done wrong.
2*. When I fail to do something well at the lesson, I shift the blame on 4.50 4.50
the others.
3. When I come across some problem, I try to recollect similar situations 4.20 4.18
from my experience, which will help me solve it.
4. Prior to the lesson, I think about the difficulty I might encounter during 4.15 4.13
the lesson. At the end of the lesson, I compare my prediction with the
reality.
5. After each conducted lesson, I analyze whether I have achieved the set 4.16 4.16
goal.
6. Communication with my colleague and then analyzing this discussion 4.03 4.05
helps me to clarify some professional issue.
7. Writing and then analyzing my writing helps me to clarify some 4.08 4.07
professional issue.
8. Judging by the students’ behavior, I always try to find out if the students 4.41 4.44
liked the lesson or not, which activity they liked, and why, which activity
they did not like, and why.
9. I try to put myself in students’ place in order to analyze if the lesson 4.02 4.02
was useful or not.
10. I always think about the conducted and future lessons even in the 4.11 4.13
moments when I am busy or at rest.
11*. I do not like observing my colleagues’ lessons, analyzing, evaluating 3.97 4.00
and comparing them with my experience.
12*. I do not like when my colleagues observe my lessons and share their 3.87 3.90
impressions and recommendations with me, which I analyze and consider
in the future.
13. I like reading and analyzing professional articles and books, comparing 4.00 3.98
them with my experience.
1. I am well versed in professional terminology. 3.87 3.91
2*. Sometimes I understand the term, but I cannot explain it to the 3.98 3.95
audience.
3. I can orally express my opinion concisely and clearly (when speaking 4.32 4.32
with students and colleagues).
4*. I find it difficult to express my opinion in written form concisely and 3.96 3.99
clearly (with my students and colleagues)
5*. I do not like discussing professional issues with my colleagues. 3.90 3.92
6. I understand students’ arguments and I try to consider them. 4.00 3.97
7. I do not find it difficult to explain the lesson aims, activity procedures 4.12 4.13
and assessment principles, etc. to my students.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
294 8. I do not find it difficult to prepare a professional conference topic. 3.45 3.44
9*. I am reluctant to participate in professional meetings/discussions. 4.03 4.06
10. I do not find it difficult to make an unprepared speech in meetings/ 3.30 3.30
discussions on pedagogical topics using theoretical knowledge.
1. I try to rely on the most recent achievements in subject methodology 3.90 3.90
and all related disciplines (Pedagogy, Psychology, etc.) and my
knowledge while planning the instructional process.
2. I always relate theory to practice (experience) at my lessons. 3.98 4.00
3. I do not find it difficult to plan and conduct a lesson. 4.12 4.14
4. I find it easy to present the materials taking into consideration the 4.05 4.04
students’ individual characteristics and I help the learners to get involved
in the instructional process.
5. If I do not know the answer to a student’s question, I admit it, and I 3.95 3.93
promise to explore it and give an answer to them at the next lesson (I do
so).
6. In the first place, I evaluate myself, I am critical of myself and then 4.11 4.10
towards my students.
7. I think I evaluate students’ knowledge objectively. Since I design 4.30 4.30
assessment criteria together with my students, my evaluation does not
cause protest.
8. I rely only on my own opinions and experience while teaching and I 4.25 4.27
think that is right.
9*. I never plan my own lessons. Instead, I often follow the plans in a 4.35 4.36
teacher’s book.
10*. I never (hardly ever) use pair and group work as they cause chaos. 4.50 4.52
11*. I never (hardly ever) use technical aids/technologies since their 4.70 4.72
integration into the instructional process will take a lot of time.
12*. I do not often consider a student’s opinion as I think I know better 4.15 4.18
what is necessary and useful for my learners.
13*. If I do not know the answer to a student’s question, I say that it is not 4.20 4.23
important, or we have no time for talking about it.
14*. I often find it difficult to manage discipline at the lesson. 3.40 3.38
*15. My students are often dissatisfied with my assessment of their 4.70 4.73
knowledge/skills.
Variable 1. Pearson correlation 1 1.000
Sig (2-tailed) .000
N 35 34
Variable 1. Pearson correlation 1.000 1
Sig (2-tailed) .000
N 34 37

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Received: January 22, 2021 Accepted: April 02, 2021 295
Cite as: Tkavashvili, E. (2021). The impact of teacher professional diary on their reflective,
communicative and professional skills. Problems of Education in the 21st Century, 79(2),
273-295. https://doi.org/10.33225/pec/21.79.273
Eka Tkavashvili Master of Education (M.Ed.), Ilia State University, 3/5 Cholokashvili Avenue,
0162 Tbilisi, Georgia.
E-mail: eka_tkavashvili@iliauni.edu.ge

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
296
REVIEW OF THE STATE OF
METHODOLOGICAL TRENDS IN OPEN
AND DISTANCE LEARNING LITERATURE
2009-2018
Scholastica Chizoma Ukwoma
University of South Africa & University of Nigeria, South Africa
E-mail: ukwomsc@unisa.ac.za
Patrick Ngulube
University of South Africa, South Africa
Abstract
Methodological trends are a key to the development of research in open and distance learning as a cognitive
field. The purpose of this study was to review the methodological trends in the open and distance learning
scholarly communication literature. It is important that research strategies are reviewed often to ensure
the production of quality research output in a cognitive discipline. The study adopted qualitative content
analysis to a sample of 1,393 articles published in five open and distance learning (ODL) journals from
2009 to 2018. The main findings of the study indicated that qualitative and quantitative methodologies
were used more often than mixed methods research methodology, which was underrepresented. This
implies that research in ODL relied heavily on positivist and interpretivist epistemologies, rather than
pragmatist ones. Survey research approaches dominated in the research articles published in the ODL
journals during the period under study. The main data analysis techniques were statistical and narrative
techniques. Having ascertained that there is a gap in the use of mixed methods research in ODL, there
is a need to set the future direction for open and distance learning research to focus on mixed methods
research to minimise the weakness inherent in using only one method.
Keywords: data analysis technique, methodological trends, open distance learning, research paradigm
Introduction
The continuous evolution and development of distance education are dependent on

research (Asdaque, 2019), which is the vehicle through which knowledge is spread and made
accessible. Research is a search for knowledge and a discovery of hidden truth, and through
research, progress is made in a specific field (Gounder, 2012). Open and distance learning
education (ODL) has grown tremendously around the world (Kundu, 2014); it is one of the most
rapidly growing fields of education in recent times (Oladejo & Gesinde, 2014) and universities
across the world are increasingly adopting it in order to increase access to university education
(Chawinga & Zozie, 2016). Since the mid-1990s, the digital transformation has changed the face
of open and distance education as we had known it (Zawacki-Richter & Qayyum, 2019). This
growth is attributed to changes brought about by information and communication technologies
which have kept pace with these developments (Gökmen, et al., 2017). There is no doubt that
different subject areas continue to emerge in the ODL field, which has contributed to the growth
of the discipline.
Research that may produce dependable and valid knowledge should be systematically
and methodologically sound (Ngulube & Ukwoma, 2019). Selecting the best set of theoretical

Scholastica Chizoma UKWOMA, Patrick NGULUBE. Review of the state of methodological trends in open and distance learning
literature 2009-2018
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
approaches, data collection methods, and analysis techniques for a project is critical to the success 297
of any research (Galport & Galport, 2015). Choosing the appropriate methodology contributes a
lot to the dependability of the study. The methodology covers the entire organization of research
activity, any research must be methodologically substantiated (Lamanauskas, 2020). Although
no methodology is superior to another, different methodologies merely aim to reach different
goals and produce different types of data (Basson & Prozesky, 2015). Therefore, researchers
should be mindful of the methodology to adopt in a research considering the wide array of
research methods that are available. Conducting research requires a systematic approach,
from presenting the problem statement and the research questions, to selecting the appropriate
research design. Research design is an important aspect of the research procedure into problem
solving and the discovery of new knowledge. Although the type of research design to adopt
in a work depends on the nature of the study. Scandura and William (2000) recognised that
differences in researchers' training may affect their choice of research design. This explains why
different disciplines adopt different research designs. Basson and Prozesky (2015) expressed
concern that, within the discipline, an over-emphasis on one type of method could be indicative
of a lack of skill in other types of methods, leading to certain topics being ignored or explored
inappropriately. This implies that researchers may tend to rely mostly on the methodologies
they are conversant with, irrespective of its appropriateness.
Research methodology explores the research phenomenon and covers the rationale in
choosing the methods and research strategies (Ngulube & Ukwoma, 2019). Some studies may
use more than one methodology (Berge & Mrozowski, 2001) because all research methods
have strengths and weaknesses (Ngulube, 2019). Quantitative, qualitative, and mixed methods
research are the three major methodologies that are associated with positivism, interpretivism
and pragmatism, respectively (Ngulube & Ukwoma, 2019; Zawacki-Richter et al., 2009).
Quantitative research is based on the measurement of quantity or amount while qualitative
research is concerned with a qualitative phenomenon involving quality (Gounder, 2012). A
methodological review of ODL research will establish field-specific theoretical bases, produce
policies, develop applications, and enhance the quality and functionality of distance education
systems (Gökmen et al., 2017). It will also contribute to the body of knowledge in the discipline
and research development in general.
Researchers have added knowledge to research reviews and methodological trends.
Zawacki-Richter et al., (2009) presented the status of distance education, the subject areas,
authorship pattern and research methodology from 2000 to 2008. Berge and Mrozowski (2001)
reviewed journals in distance education between 1990 and 1999 to ascertain the research trends
in the articles. Bozkurt et al., (2015) examined the trends in distance educational research using
a content analysis of research articles published between 2009 and 2013. Zawacki-Richter and
Naidu (2016) mapped research trends in the journal Distance Education. A comparative study
of research articles published in 2005 and 2015 in seven peer-reviewed journals on ODL was
carried out by Wong et al., (2016). Other studies on methodological trends in distance education
are the review of methodological trends in the distance education theses published in Turkey
from 2005 to 2014 (Gökmen et al., 2017). Asdaque (2019) explored the research trends in
doctoral research in distance education in Pakistan.
Reviewing the methodological trend after the study of Zawacki-Richter et al., (2009)
which covered 2000 to 2008 in five ODL journals, will be helpful to ascertain the patterns in
the trends a decade later. Although, Bozkurt et al. (2015) covered 2009 to 2013 in seven ODL
journals, the present study followed the line of Zawacki-Richter et al., (2009) in covering a
period of ten years in five core ODL journals. It is highlighted by Scandura and William (2000)
that an examination of methodological trends may provide insights into the possible future
development of research methodology in a field. The findings of this study will help to ascertain
if there is any new development in the methodological trends in ODL and provide insight into
the methodological trends adopted in ODL journals.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
298 Problem Statement
Reviewing the research methods used in a discipline helps to keep researchers abreast
of the trends and areas for further research. As Ullaha and Ameen (2018) stated, knowledge of
methods used in a discipline is invaluable for researchers who want to choose among appropriate
methods in conducting reliable and valid research. Similarly, Gökmen et al., (2017) opined that
studying methodological trends of ODL research will enhance the quality and functionality
of distance education systems. Although extant studies on methodological trends in ODL as
highlighted above reviewed the trend in ODL theses and research literature as published in
ODL journals, it is not certain if ODL literature still maintains the same methodological trend
as identified by Zawacki-Richter et al., (2009) considering the period the study was conducted
with the advancements and developments that have taken place in ODL as a discipline. This is
also in view of the fact that ODL education has continued to grow tremendously (Kundu, 2014).
To ascertain the status of the methodological trend, ODL literature must be studied, and this is
the gap this study seeks to narrow. The study is significant because it provides insights into the
methodological trends adopted in ODL journals, and the diversity of approaches used which
will be an asset to ODL as a discipline and researchers in general.
Classification of Methodological Indicators
Employing an appropriate methodology gives credence to a study, and this informs why
reviewing research methodology in any discipline from time to time is necessary to strengthen
its research. In the context of management research, Scandura and Williams (2000) highlighted
that it is important for researchers to assess the methods they employ for the field to progress.
This is applicable to other disciplines as the choice of research methodology, appropriateness
of data collection and analysis methods employed have a serious impact on the conclusion of
the research.
Choosing the appropriate research design in a study is important to produce a reliable
and trustworthy result, which contributes a lot to the impact of the study. Such studies are
always referenced because it is believed that the proper research designs were employed. The
research design model by Ngulube (2019) showed that there has not been agreement on the
use of research terminologies. The author highlighted that research design comprises research
paradigms, methodology, ethical concerns, research methods, and data collection and analysis
methods. Saunders et al., (2016) also explained research design as a general plan of answering
your research question(s). The analogy by Saunders et al., (2016) indicates that research
follows a systematic approach with each stage linking to the other. There have been many
misconceptions of what constitutes research methodology, and the proper use of the research
methodology terms, as could be seen from writers. Daniel (2016) classifies research designs
as either qualitative, quantitative research or mixed methods. Gounder (2012) attempts to
differentiate between research methods and research methodology. Research methods involve
conducting experiments, tests, surveys, and the like, while research methodology involves
the learning of the various techniques that can be used in conducting research and tests,
experiments, surveys, and critical studies. Apuke (2017) categorised quantitative methods
into survey research, correlational research, experimental research, and causal-comparative
research. There is a need for proper distinction of these terms in research writing and the proper
terms used to avoid confusion.
Research methodology deals with the explanation of why a research study is undertaken,
how one formulates a research problem, what types of data were collected, what method has
been used and why a specific technique of data analysis was used (Gounder, 2012). Following
the line of argument of Ngulube (2019), research design, as used in the context of this study,

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
comprises the totality of the research processes starting from the philosophical assumptions. 299
Research methodologies are qualitative, quantitative, and mixed methods research, which
translates into the qualitative, quantitative, and mixed methods research approaches/strategies
such as survey, case study, and grounded theory, and so on. Research techniques include data
collection methods and samples, the data collection method to be adopted depends on the nature
of the research. It is necessary to define these methodological indicators as used in this study
because one of the major problems that arise when conducting methodological trends is the fact
that the indicators are presented in a wide variety of ways.
Different phenomena may require the use of different methodologies. The methodology
employed must match the phenomenon of interest (Krauss, 2005). Qualitative research methods
are commonly used when there is little current understanding of a complex phenomenon or if
current knowledge is fragmented (Kyngäs, 2020). They are highly appropriate for questions
where pre-emptive reduction of the data will prevent discovery (Ochieng, 2009). Qualitative
researchers use phenomenological techniques and their worldviews to extract meaning, and
quantitative researchers use an array of statistical procedures and generalisations to determine
what their data mean (Onwuegbuzie & Leech, 2005). Quantitative research deals with
quantifying and analysing variables in order to get results (Apuke, 2017; Rana & Sharma, 2016).
Positivists (i.e., quantitative) believe in a single reality that can be measured reliably and validly
using scientific principles, and interpretivists (i.e., qualitative) believe in multiple constructed
realities that generate different meanings for different individuals, and whose interpretations
depend on the researcher’s lens (Onwuegbuzie & Leech, 2005). Any research method chosen
will have inherent flaws, and the choice of that method will limit the conclusions that can
be drawn (McGrath, 1982; Scandura & William, 2000). The realisation that quantitative and
qualitative methodologies have advantages and disadvantages led to the emergence of mixed
methods (MM) first in the form of triangulation or multi-methods leading to mixed methods
research (MMR) (Ngulube & Ukwoma, 2019).
Each methodology is associated with a specific research approach. Quantitative
approaches include survey, experimentation, and case study. Creswell (2013) highlighted that
researchers may choose several qualitative approaches, such as narrative research, grounded
theory, case study, ethnography, and phenomenology. Mixed methods research approaches
include basic (i.e., sequential, and concurrent) and advanced (i.e., multistage) approaches. The
research approach employs various data collection methods. Qualitative data instruments such
as observation, open-ended questions, in-depth interviews (audio or video), and field notes
are used to collect data from participants in their natural settings (Daniel, 2016), and artefact
analysis, including document reviews. These data collection instruments provide the researcher
with an opportunity to probe and obtain more factual information from the respondents. In
qualitative, quantitative or MMR, the corresponding data collection instruments should be
employed. Surveys and scales are mostly used in the quantitative methodology while the
interview method is mostly used in qualitative methodology (Gökmen et al., 2017).
Another important step in the research process is analysis of data. Data can be analysed
using narrative or statistics, it depends on the nature of data and the methodology adopted.
We contend that researchers need to utilise at least two types of data analysis tools in order to
triangulate results (Leech & Onwuegbuzie, 2007). Descriptive statistics were the most applied
data analysis tools (Asdaque, 2019). The choice of statistical or narrative data analysis depends
on the research approach and the nature of data. Data can be analysed using computer assisted
data analysis software; they help in manipulating the data to put them into different formats.
Ngulube and Ukwoma (2019) highlight that data can be analysed manually or using data analysis
software such as Microsoft Excel® and SPSS® for quantitative data, and NVivo, Altlas.ti®,
for qualitative data. Silver and Lewins (2014) identify ATLAS.ti, Dedoose, MAXQDA, NVivo,
QDA Miner, and Transana, as some of the computer assisted software in analysing qualitative

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
300 data. The use of these software programs depends on the type of data to analyse and the result
expected by the researcher. Although they are tools that assist in data analysis, the choice of
software to use and the actual interpretation depend on the researcher (Lewins & Silver, 2009).
As in the case of previous studies, Berge and Mrozowski (2001), Bozkurt et al., (2015),
Wong, et al., (2016), Zawacki-Richter and Naidu (2016) and Zawacki-Richter et al., (2009),
the current study used articles in peer-reviewed journals for analysis. Article is one of the
forms of introducing and making public the results of research activities (Lamanauskas, 2019),
the articles are published in peer reviewed journals. Peer-reviewed journals are important
channels for the dissemination of the products of research and scholarship (Zawacki-Richter &
Anderson, 2011). Journals and articles are key resources for the diffusion of scientific knowledge
Bozkurt (2019 b), because they are subjected to some degree of vetting for quality (Ngulube &
Ukwoma, 2019). Hence, the choice of reviewing methodological trends in journals over theses
and dissertations. In ODL, there are several journals that publish peer-reviewed articles. These
journals have been publishing core ODL research over the years; they are indexed in prominent
educational databases and have made tremendous contributions to the field of ODL. The goal
of ODL is to make learning open and accessible to everyone (Van der Merwe & Van Heerden,
2013).
Research Questions
The general purpose of the research was to review the methodological trend of ODL
research literature from 2009 to 2018 while the specific research questions were as follows:
1 What research methodologies were employed, and which methodology was the
most prevalent in the ODL journals within the period under study?
2 Which research approach is most prevalent in the research articles in ODL journals,
and what is the trend in the use of the research approaches?
3 Which data collection techniques were commonly employed in the research articles
published in ODL journals?
4 What data analysis technique is employed, and which data analysis software was
mostly used from 2009 to 2018?
General Background
The study adopted a qualitative content analysis technique of the research articles
published in five major ODL journals (see Çakıroğlu et al., 2019) from 2009 to 2018. Content
analysis is the most significant method used to examine methodological and publication trends
(Umer & Razi, 2018). The choice of ten-year period is necessary to ascertain the methodological
trends in ODL research journals. Berge and Mrozowski (2001) covered a period of ten years,
Umer and Razi (2018) covered a period of 28 years, Henry et al, (2016) covered a 30-year period.
Therefore, studies cover varying periods to determine the trends in scholarly communication,
and the changing pattern of scholarly communication can be measured in five-year periods
(Stansbury, 2002). Since this study is using the lens of Zawacki-Richter et al., (2009), which
covered 2000-2008, it implies that the starting point of 2009 should be adopted for this study
with a cut-off date of 2018, to determine the trend for at least a decade.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Sample 301
The core ODL journals studied were, The American Journal of Distance Education
(AJDE), Distance Education (DE), The European Journal of Open, Distance and e-Learning
(EURODL), Open Learning: The Journal of Open, Distance and e-Learning (OL) and The
International Review of Research in Open and Distributed Learning (IRRODL). These journals
were identified by Çakıroğlu et al., (2019) as core ODL journals, which publish core open
and distance education literature. In addition, their articles are written in English language.
Since the study done by Çakıroğlu et al., (2019) is still a recent study and these journals are
internationally recognised journals, searchable in leading educational databases such as the
Social Sciences Citation Index (SSCI), Scopus, Education Resources Information Center
(ERIC), Directory of Open Access Journals (DOAJ) and Elton B Stephens Co. (EBSCO), this
study also adopted those journals as the sample frame.
Instrument and Procedure
The research articles were downloaded from the journal home page, excluding book
reviews, reflections, and editorial comments. The data were cleaned up and relevant information
needed for the study, including the methodology, research approach, data collection techniques,
and method of data analysis and data analysis software used, was extracted. This phase lasted
for a period of three months (August to October 2019). Out of 1,393 articles, 1,119 (80%)
were empirical articles, while 20% were non-empirical articles (opinion papers). The study was
limited to 1,119 articles that were empirical.
For the intercoder reliability, a sample of 25 articles, five randomly selected from each
of the five journals, were coded by an independent coder (the articles were coded separately by
the two authors, they later exchanged their coding for crosschecking) to ensure that it was not
based on the personal decision, although no statistical test was done for intercoder reliability.
There were some cases where the descriptions were not correct; such errors were corrected. The
elements identified from the articles for the study were: the methodologies, research approaches/
strategies, data collection techniques, data analysis method and data analysis software used
such as such as Microsoft Excel® and SPSS® for quantitative data, and NVivo, Altlas.ti®, for
qualitative data.
Data analysis
The coded data were entered on an Excel spreadsheet and analysed. Descriptive statistical
methods comprising frequency and percentage were used to present the results. Tables were
used in presenting the results based on the research questions.
Research Results
Research methodologies employed and most prevalent in the ODL journals within the
period under study are presented in Table 1.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
302 Table 1
Research Methodologies Employed in the ODL Journals in the Period under Study
Non-
No. of Empirical Qualitative Quantitative MMR
empirical
Name of journal articles articles
articles
n/(%) n/(%) n/(%) n/(%) n/(%)
n/(%)
The American Journal of
175 30 (17) 145 (83) 82 (57) 55 (38) 8 (5)
Distance Education (AJDE)
Distance Education (DE)

207 36 (17) 171 (83) 87 (51) 80 (47) 4 (2)
The European Journal
of Open, Distance and 165 38 (23) 127 (77) 80 (63) 45 (35) 2 (2)
e-Learning (EURDOL)
The International Review
of Research in Open
676 108 (16) 568 (84) 283 (50) 271 (48) 14 (2)
and Distributed Learning
(IRRODL)
Open Learning: The Journal
of Open, Distance and 170 62 (36) 108 (64) 58 (54) 49 (45) 1 (1)
e-Learning (OL)
Total 1393 274 (20) 1 119 (80) 590 (53) 500 (44) 29 (3)
It can be seen that out of the 1,393 articles that formed the research framework, 1, 119
(80%) were empirical while 20% were non-empirical. ODL research employed qualitative
(53%), quantitative (44%), and mixed method research (3%) within the period under study.
Looking at the trend in each of the journals studied, it is clear that qualitative, quantitative, and
mixed method research methodologies were employed.
Qualitative and quantitative methodologies were prevalent. The use of mixed method
research has not made much of an inroad into ODL research. It is believed that the researchers
rely on using scientific principles in gathering their data and interpreting them, which is more
like the principles of positivist and interpretivist. This could account for why they were prevalent
in ODL research. Similarly, in the individual journals, qualitative and quantitative approaches
were prevalent. It is only in AJDE that mixed methods were recorded as 5% and the others were
recorded as 2% and 1%.
The research approaches most prevalent in research articles in ODL journals and the
trend in the use of the research approaches are illustrated in Table 2.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Table 2 303
Prevalent Research Approaches Employed in the Period under Study
Year 2009 2010 2011 2012 2013 2014 2015 201 6 2017 2018
Total
Approach n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%)
598
Survey 19 (3) 20(3) 29(5) 38(6) 86(14) 88(15) 68(11) 80(13) 95(16) 75(13)
244
Case study 28(11) 17(7) 27(11) 42(17) 14(6) 17(7) 18(7) 32(13) 23(9) 26(11)
111
Content analysis 8(7) 6(5%) 9(8) 10(9) 6(5) 11(10) 20(18) 15(14) 10(9) 16(14)
89
Experimental 15(17) 3(3%) 3(3) 10(11) 8(9) 10(11) 10(11) 12(14) 11(13) 7(8)
20
Design-based study - 1(5%) 4(20) 3(15) 1(5) 1(5) 3(15) - 5(25) 2(10)
16
Ethnographic 1(6) 1(6) 1(6) - 3(19) 4(25) 1(6) 3(19) - 2(13)
Action 15
- 1(7) 1(7) 3(20) - 2(13) 2(13) - 5(33) 1(7)
research
10
Bibliometric study 1(10) 2(20) - - - - 2(20) - 3(30) 2(20)
Phenomenological 9
- - 1(11) - - 2(22) 3(33) 1(11) 1(11) 1(11)
study
7
Grounded theory - 3(43) - - - - - - 3(43) 1(14)
Total 72 54 75 106 118 135 127 143 156 133 1 119

Note: * Decimal points were rounded off to the nearest to get a round figure
The trend in the use of research approaches over the years shows that survey, case study,
content analysis and experiments were employed in ODL research articles each year. It can
be seen that 15 research approaches were identified, out of which four were prevalent. The
prevalent research approaches employed in ODL research literature within the study period
were survey (53%), followed by case study (22%), content analysis (10%), and experimental
(8%). Although other research approaches were employed within the period, they were not high
in frequency. This shows that the use of qualitative and quantitative research approaches was
prominent in ODL research literature.
Data collection techniques commonly employed in the research articles published in
ODL journals are presented in Table 3.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
304 Table 3
Data Collection Techniques Employed Within the Period
Data collection
document Online group chat
Questionnaire Interview Obser-vation Datasets
analysis and discussion
n/(%) n/(%) n/(%) n/(%)
n/(%) n/(%)
Journals
76 39 43 7 9 5
AJDE
(13) (11) (15) (10) (15) (13)
120 73 47 7 9 2
DE
(20) (21) (16) (10) (15) (5)
74 44 31 15 19 3
EURDOL
(12) (12) (11) (22) (33) (8)
269 158 142 30 16 28
IRRODL
(45) (45) (48) (44) (28) (72)
59 38 31 9 5 1
OL
(10) (11) (11) (13) (9) (2)
Total 598 352 294 68 58 39
The results on Table 3 is a case where the percentage is more than 100% because
many articles used more than one data collection technique. It shows that 53% of the articles
used questionnaire as data collection technique, followed by interview (31%), and document
analysis (26%). Others were observation (6%), chats and discussions (5%), and data sets (3%).
Similarly, in the individual journals, questionnaires and interviews were predominantly used
more than other data collection techniques; in IRRODL questionnaire scored 269, interview
158, and document analysis 102; in DE questionnaire recorded 120, interview 73, and document
analysis 33; in AJDE questionnaire recorded 76, interview 39 and document analysis 23; in OL
questionnaire recorded 59, interview 38, and document analysis 17; In EURDOL questionnaire
recorded 74, interview, focus group recorded 44, and document analysis 22.
Data analysis techniques employed and the most data analysis software used since 2009-
2018 are illustrated in Tables 4 and 5 respectively.
Table 4
Data Analysis Techniques Employed in the Study Period
Year 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Data Total
analysis
method n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%)
Statistical 59 42 60 69 110 110 100 115 128 98 891

analysis (7) (5) (7) (8) (12) (12) (11) (13) (14) (11)
Thematic 20 17 21 31 19 28 27 35 25 36
259
analysis (8) (6) (8) (12) (7) (11) (10) (14 (10) (14)
Phonological 1 1 2
- - - - - - - -
analysis (50) (50)
Total 79 59 81 100 129 139 125 151 153 134 1150

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
To analyse the results of the data analysis techniques employed in the ODL research as 305
presented on table 4, articles that employed inferential, descriptive, social network analysis,
factor analysis, regression in data analysis irrespective of those that used software and those
that did not, were grouped under statistical analysis. Content analysis, narratives and thematic
analysis were grouped together. In addition, there were some studies that employed both
narrative and statistical analysis in data analysis. The results indicated that the majority (72.4%)
of the articles employed statistical analysis and 25.3% employed narrative in data analysis,
although many of them did not mention the software employed in data analysis. The presence of
different statistical analyses in ODL literature helps in manipulating and presenting the research
results using different formats.
Table 5
Trends in the Use of Data Analysis Software
Year
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Data Total
analysis
n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%) n/(%)
software
7 9 11 19 30 34 20 34 27 17
SPSS 208
(3) (4) (5) (9) (14) (16) (10) (16) (13) (8)
2 2 3 1 1 9 3 10 5 6
NVivo 42
(5) (5) (7) (2) (2) (21) (7) (24) (12) (14)
1 2 1 3 1 2 1
Atlas ti - - - 11
(9) (18) (9) (27) (9) (18) (9)
1 1 1 1 2
Excel - - - - - 6
(17) (17) (17) (17) (33)
MAXQDA - - - - - 1 - - - 2 3
SAS JMP - - - - - - 1 - - - 1
AMOS - - - - - - - - - 1 1
Total 11 12 17 21 35 46 25 46 33 16 272
The results (Table 5) indicated that out of the 1,119 empirical articles, 272 (24,3%)
articles mentioned the software used in data analysis. Although many of the studies used different
statistical methods in analysing their data, they failed to mention the software employed in data
analysis. This accounted for the low number of software as identified from the study. From the
results, it could be seen that most of the studies in ODL research prefer SPSS data analysis
software followed by NVivo. These are the quantitative and qualitative data analysis software.
Regarding the data analysis software used, the highest number of studies that used data
analysis software was recorded in 2014 and 2016 with 46 each, followed by 2013 and 2017
with 35 and 34, respectively. The lowest number was recorded in 2010 (12). Further analysis of
why 2010 recorded the lowest frequency in the research approaches and data analysis software
used indicated that the number of articles for 2010 was lower when compared to other years. In
addition to that, over the years, there has been an increase in the use of SPSS and NVivo data
analysis software, except in 2015 and 2018 where SPSS had a decrease and NVivo in 2015,
2017 and 2018 there was a decrease in the number of articles that used it in data analysis.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
306 Discussion
The choice of qualitative and quantitative methodologies in ODL research could be

attributed to the multidimensional interaction of the discipline with living and non-living
entities Bozkurt (2019b), and the use of interactive communication technologies forming the
learning environment in ODL (Çakıroğlu et al., 2019). Knowledge of these technologies and
users’ attitudes towards them are important for effective implementation and success in the
field, as they are required in predicting the future. Hence the predominant use of qualitative
research methodology, which is commonly used when there is little current understanding of
a complex phenomenon (Kyngäs, 2020), quantifying and analysing variables in order to find
results Apuke (2017). Positivists believe in a single reality that can be measured reliably and
validly using scientific principles, and interpretivists believe in multiple constructed realities
that generate different meanings for different individuals, and whose interpretations depend
on the researcher’s lens (Onwuegbuzie & Leech, 2005). There are few articles that employed
pluralist ontologies. ODL cuts across many disciplines, as such, mixed method research would
be helpful to the discipline. Mixed methods research is a dynamic option for expanding the
scope and improving the analytical power of studies (Sandelowski, 2000), combining the saving
grace of both quantitative and qualitative methods (Taipale & Fortunati, 2014). Some scholars
don’t acknowledge the richness of both traditions, both qualitative and quantitative research can
be used to address almost any kind of research question (Ochieng, 2009). The prevalence of
qualitative and quantitative methodologies in ODL was also discovered in the study of Bozkurt
et al., (2015) and the Brazilian distance education publications between 1992 to 2007 were
predominantly qualitative (Dutra De Oliveira Neto & Santos, 2010).
Qualitative and quantitative research approaches such as survey, case study, document
analysis, and experiments, were prevalent in the research literature studied. Survey seems to be
a dominant research approach in many disciplines, the use of survey was noted in (Asdaque,
2019), other studies noted the dominance of the survey approach in research (Gökmen et al.,
2017; Ngulube & Ukwoma, 2019; Rana & Sharma, 2016). Although one may be tempted
to accept the statement of Basson and Prozesky (2015) on the reason for over-reliance on
one approach, it may be difficult to accept that conclusion from this study. Using different
approaches in research ensures reliability and gives credence to the result. Case study is mostly
used in qualitative and mixed methods research (Gökmen et al, 2017). In the study of the use
of the case study research method for analysing MOOCs, the quantitative research method
was more prevalent than the interpretivist paradigm (Montes-Rodríguez et al, 2019). Berge
and Mrozowski (2001) highlighted that correlational research was seldom utilised in distance
education, although it was employed in a negligible number of articles, as identified in this study.
Ascertaining the perceptions and interest of the user community is key to the development of
ODL programmes and to making informed decisions. Therefore, questionnaires and interviews
are the easiest techniques to use to interact and collect reliable information. Daniel (2016)
stated that they are used to collect data from participants in their natural settings. No wonder
Lamanauskas (2020) stated that Questionnaires and tests are probably the most common types
of instruments used in quantitative research.
The predominant use of statistical analysis technique could be traced to the quantitative
methodology dominant in the ODL literature. The use of thematic and phenomenological
analysis was predominantly low in ODL research articles studied in the period under study.
Neubauer et al, (2019) stated that phenomenological research broadens the understanding of
the complex phenomena involved in learning, behaviour, and communication that are germane
to the field.
Comparing the results of Zawacki-Richter et al., (2009), which recorded quantitative
(29.1%), qualitative (19.9%) and triangulation (mixed methods) (12.9%), with the present

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
study, a shift was noted in that qualitative recorded 53%, quantitative recorded 44%, and mixed 307
methods recorded 3%. This implies that between 2009 and 2018, qualitative and quantitative
methodologies were preferred in ODL research literature. Mixed methods research was reduced
to below two digits. It is worthy to mention that there is a slight change in the pattern of
methodology used in ODL research. As new innovations and subject areas emerge, it will affect
the methodologies and approaches of analysing them. Therefore, ODL research is witnessing
a gradual change in its methodology, but pragmatist epistemology is yet to be dominant in
the research literature. Although pragmatist epistemology is not yet dominant that does not
mean that ODL research is not methodologically sound, different subject areas require different
methodological approaches. No methodology is superior to another; different methodologies
aim for different goals and produce different types of data (Basson & Prozesky, 2015). However,
researchers are being called upon to use mixed methods research so that they may be able to
comprehensively understand the phenomenon under study, and to answer the ‘who’, ‘how’,
‘what’, where’, ‘whom’ and ‘why’ questions in a single study.
The state of the methodological trends of ODL research shows that qualitative,
quantitative, and mixed methods approaches were employed in ODL research. Qualitative and
quantitative methods were predominantly used in the study period. The presence of positivist
and interpretivist epistemology is also reflected in the research approaches such as survey and
case study that were used. Questionnaire, interview, and document analysis were the most
employed data collection techniques, a greater percentage of the articles employed statistical
data analysis techniques. Research methodology is an important aspect of research procedure
into problem solving and discovery of new knowledge although the methodology to adopt in a
study depends on the nature of the problem.
This study has implications for researchers and the ODL discipline. This study will serve
as a reference for other researchers who are interested in methodological trends in ODL and
provide a guide for further research.
Recommendations
Using statistical analysis helps to present and manipulate the research data in a simple
format that could be read at a glance. It also helps to present the findings using varying formats.
But narrative tends to probe more for an in-depth knowledge, researchers should be encouraged
to use a combination of methods in data analysis.
Adopting phenomenological and thematic method in data analysis may be necessary as
ODL is a multidisciplinary field. Using varied data analysis methods will give researchers the
opportunity to explore the ODL phenomenon in a different context.
Research keeps on evolving and new areas continue to emerge. Adopting other research
approaches apart from survey which has gained much prominence, may provide a different
result and meaning to the research.
Considering that ODL is concerned with multifaceted problems, using monomethods
may fail to deal with the problem effectively. It will be worthwhile for ODL researchers to
adopt mixed methods more and more. Mixed methods research combines the advantages of
both qualitative and quantitative methods to provide a comprehensive picture of a phenomenon.
For the researchers, using a combination of research methods will be helpful in unravelling
in-depth information from the participants, which helps researchers come up with enough
evidence. The prevalence of qualitative and quantitative methodology over mixed methods
may be attributed to the fact that they are the earliest methodologies known to researchers. Even

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
308 though different phenomena require different methodologies, most researchers tend to adopt the
method they are familiar with and have enough knowledge. Introducing researchers early to
mixed methods research will be helpful.
Limitations and Further Research
As no study is without limitations, this study is not an exception. This study could not
establish the reasons for the decline in research approach and data analysis software used at
certain periods, as observed in the study. A different methodological approach has the potential
to provide answers to these unanswered questions. The review of the methodological trend is
restricted to papers published between 2009 and 2018, and the result cannot be generalised for
the whole ODL research literature.
References
Apuke, O. D. (2017). Quantitative research methods a synopsis approach. Arabian Journal of Business
and Management Review (Kuwait Chapter), 6(10), 40-45. https://doi.org/10.12816/0040336
Asdaque, M. M. (2019). Exploring the research trends-related imbalances in doctoral level research on
distance education in Pakistan. Pakistan Journal of Distance & Online Learning, V (I), 105-122.
https://pjdol.aiou.edu.pk/wp-content/uploads/2019/07/8-exploring-research-trends.pdf
Basson, I., & Prozesky, H. E. (2015). A review of methodological trends in South African Sociology,
1990–2009. South African Review of Sociology, 46(3), 4-27. https://doi.org/10.1080/21528586
.2015.1022595
Berge, Z. L., & Mrozowski, S. (2001). Review of research in distance education, 1990 to 1999. American
Journal of Distance Education, 15(3), 5-19. https://doi.org/10.1080/08923640109527090
Bozkurt, A. (2019a). From distance education to open and distance learning: A Holistic evaluation of
history, definitions, and theories. In S. Sisman-Ugur, & G. Kurubacak (Eds.), Handbook of
research on learning in the age of transhumanism (pp. 252-273). IGI Global.
Bozkurt, A. (2019b). Intellectual roots of distance education: A progressive knowledge domain analysis.
Distance Education, 40(4), 497–514. https://doi.org/10.1080/01587919.2019.1681894
Bozkurt, A., Akgun-Ozbek, E., Yilmazel, S., Erdogdu, E., Ucar, H., Guler, E., Sezgin, S., Karadeniz,
A., Sen-Ersoy, N., Goksel-Canbek, N., Dincer, G. D., Ari, S., & Aydin, C. H. (2015). Trends in
distance education research: A content analysis of journals 2009-2013. International Review of
Research in Open and Distributed Learning, 16(1), 330-363.
Çakıroğlu, Ü., Kokoç, M., Gökoğlu, S., Öztürk, M., & Erdoğdu, F. (2019). An analysis of the journey of
open and distance education: Major concepts and cutoff points in research trends. International
Review of Research in Open and Distributed Learning, 20(1), 1-20.
Chawinga, W. D., & Zozie, P. A. (2016). Increasing access to higher education through open and distance
learning: Empirical findings from Mzuzu University, Malawi. International Review of Research
in Open and Distributed Learning, 17(4), 1-20
Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd
ed.). Sage.
Daniel, E. (2016). The Usefulness of qualitative and quantitative approaches and methods in researching
problem-solving ability in science education curriculum. Journal of Education Practice, 7(15),
91-100.
Dutra De Oliveira Neto, J., & Santos, E. M. (2010). Analysis of the methods and research topics in a
sample of the Brazilian distance education publications, 1992 to 2007. The American Journal of
Distance Education, 24(3), 119-134. https://doi.org/10.1080/08923647.2010.497325
Galport, M., & Galport, N. (2015). Methodological trends in research on evaluation. In Paul R.
Brandon (Ed.), Research on evaluation. New Directions for Evaluation, 148, 17–29. https://doi.
org/10.1002/ev.20154
Ghosh, S., Nath, J., Agarwal, S., & Nath, A. (2012). Education system: Past, present and future – a
systematic study of an alternative education system. Journal of Global Research in Computer
Science, 3(4), 53-57.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Gökmen, Ö. F., Uysal, M., Yaşar, H., Kırksekiz, A., Güvendi, G. M., & Horzum, M. B. (2017). 309
Methodological trends of the distance education theses published in Turkey from 2005 to 2014:
A content analysis. Education and Science, 42(189), 1-25.
Goundar, S. (2012). Research methodology and research methods. In S. Goundar cloud computing.
https://www.researchgate.net/publication/333015026
Henry, C., Foss, L., & Ahl, H. (2016). Gender and entrepreneurship research: A review of methodological
approaches. International Small Business Journal, 34(3), 217–241 https://doi.org/10.1177/0266
242614549779
Krauss, S. E. (2005). Research paradigms and meaning making: A primer. The Qualitative Report, 10(4).
http://www.nova.edu/ssss/QR/QR10-4/krauss.pdf
Kundu, S. (2014). Open and distance learning education its scope and constraints in Indian scenario.
IOSR Journal of Humanities and Social Science (IOSR-JHSS), 19(4), 01-05.
Kyngäs, H. (2020). Qualitative research and content analysis. In Kyngäs, H., Mikkonen, K., Kääriäinen,
M. (Eds), The application of content analysis in nursing science research. Springer, Cham.
https://doi.org/10.1007/978-3-030-30199-6_1
Lamanauskas, V. (2019). Scientific article preparation: A comprehensive introduction. Problems of
Education in the 21st Century, 77(6), 688-694. https://doi.org/10.33225/pec/19.77.688
Lamanauskas, V. (2020). Scientific article preparation: Methodology description. Problems of Education
in the 21st century, 78(2), 136-146. https://doi.org/10.33225/pec/20.78.136
Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: A call for
data analysis triangulation. School Psychology Quarterly, 22(4), 557–584. https://doi.
org/10.1037/1045-3830.22.4.557
Lewins, A., & Silver, C. (2009). Choosing a CAQDAS package: A working paper by, 6th edition CAQDAS
networking project and qualitative innovations in CAQDAS project. (QUIC).
McGrath, J. (1982). Dilemmatics: The study of research choices and dilemmas, In J. E. P. McGrath, J.
Martin, & R. A. Kulka (Eds.), Judgment calls in research, Sage Publications.
Montes-Rodríguez, R., Martínez-Rodríguez, J. B., & Ocaña-Fernández, A. (2019). Case Study as a
research method for analyzing MOOCs: Presence and characteristics of those case studies in the
main scientific databases. International Review of Research in Open and Distributed Learning,
20(3), 59-79.
Neubauer, B. E., Witkop, C. T., & Varpio, L. (2019). How phenomenology can help us learn from the
experiences of others. Perspectives on Medical Education, 8, 90-97. https://doi.org/10.1007/
s40037-019-0509-2
Ngulube, P. (2010). Mapping mixed methods research in library and information science journals in Sub-
Saharan Africa 2004–2008, The International Information & Library Review, 42(4), 252-261.
https://doi.org/10.1080/10572317.2010.10762870
Ngulube, P. (2019). Mapping methodological issues in knowledge management research, 2009–2014.
International Journal of Knowledge Management, 15(1), 85-100. https://doi.org/10.4018/
IJKM.2019010106
Ngulube, P., & Ukwoma, S. C. (2019). Cartographies of research designs in library information science
research in Nigeria and South Africa, 2009-2015. Library & Information Science Research, 41(3),
Article 100966. https://doi.org/10.1016/j.lisr.2019.100966
Ochieng, P. A. (2009). An analysis of the strengths and limitation of qualitative and quantitative research
paradigms. Problems of Education in the 21st Century, 13, 13-18. http://www.scientiasocialis.lt/
pec/node/242
Oladejo, M. A., & Gesinde, A. M. (2014). Trends and future directions in open and distance learning
practice in Africa. Journal of Education and Practice, 5(18), 132-139.
Onwuegbuzie, A. J., & Leech, N. L. (2005). Taking the “Q” out of research: Teaching research
methodology courses without the divide between quantitative and qualitative paradigms. Quality
& Quantity, 39, 267-296.
Rana, S., & Sharma, S. K. (2016). A review on the state of methodological trends in international
marketing literature. Journal of Global Business Advancement, 9(1), 90–107.
Sandelowski, M. (2000). Focus on research methods combining qualitative and quantitative sampling,
Data collection, and analysis techniques in mixed-method studies. Research in Nursing & Health,
23, 246–255.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
310 Saunders, M., Lewis, P., & Thornhill, A. (2016). Research methods for business Students (7th ed.),
Pearson Education.
Scandura, T. A., & William, E. A. (2000). Research methodology in management: current practices,
trends, and implications for future research. Academy of Management Journal, 43(6), 1248-1264.
Silver, C., & Lewins, A. (2014). Using software in qualitative research. SAGE publications. http://dx.doi.
org/10.4135/9781473906907
Stansbury, M. C. (2002). Problem statements in seven LIS journals: An application of the Hernon/
Metoyer-Duran attributes. Library and Information Science Research, 24, 157–168. https://doi.
org/10.1016/S0740-8188(02)00110-X
Taipale, S., & Fortunati, L. (2014). Capturing methodological trends in mobile communication studies.
Information, Communication & Society, 17(5), 627–642. https://doi.org/10.1080/136911
8X.2013.862562
Ullaha, A., & Ameen, K. (2018). Account of methodologies and methods applied in LIS Research:
A systematic review. Library and Information Science Research, 40, 53–60. https://doi.
org/10.1016/j.lisr.2018.03.002
Umer, M., & Razi, S. (2018). Analyzing research methodologies and publication trends in service
marketing literature. Cogent Business & Management, 5(1), Article 1446265 https://doi.org/10.
1080/23311975.2018.1446265
Van der Merwe, T. M., & Van Heerden, M. E. (2013). Ease of use and usefulness of webinars in an
open distance learning environment: An activity theory perspective. In McNeill, J., & Bradshaw,
K. (Eds.), SAICSIT '13: Proceedings of the South African institute for computer scientists and
information technologists conference (pp. 262-270). Association for computing machinery.
https://doi.org/10.1145/2513456.2513481
Wong, Y. Y., Zeng, J., & Ho, C. K. (2016). Trends in open and distance learning research: 2005 vs
2015. Asian Association of Open Universities Journal, 11(2), 216-227. https://doi.org/10.1108/
AAOUJ-09-2016-0035
Zawacki-Richter, O., & Anderson, T. (2011). The geography of distance education-bibliographic
characteristics of a journal network. Distance Education, 32, 441–456.
Zawacki-Richter, O., Bäcker, E. M., & Vogt, S. (2009). Review of distance education research (2000
to 2008): analysis of research areas, methods, and authorship patterns. International Review of
Research in Open and Distance Learning, 10(6), 21-50.
Zawacki-Richter, O., & Naidu, S. (2016). Mapping research trends from 35 years of publications in
Distance Education. Distance Education, 37(3), 245-269.
Zawacki-Richter, O., & Qayyum, A. (2019). Introduction. In Olaf Zawacki-Richter & Adnan Qayyum
(Eds), Open and Distance Education in Asia, Africa, and the Middle East. (pp1-6), Springer.
https://doi.org/10.1007/978-981-13-5787-9_1

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
311
Received: October 17, 2020 Accepted: March 15, 2021
Cite as: Ukwoma, S. C., & Ngulube, P. (2021). Review of the state of methodological trends
in open and distance learning literature 2009-2018. Problems of Education in the 21st Century,
79(2), 296-311. https://doi.org/10.33225/pec/21.79.296
Scholastica Chizoma Ukwoma School of Interdisciplinary Research and Graduate Studies, University
(Corresponding author) of South Africa, Preller St, Muckleneuk, 0002 Pretoria, South Africa.
E-mail: ukwomsc@unisa.ac.za scholar.ukwoma@unn.edu.ng
Patrick Ngulube Professor, School of Interdisciplinary Research and Graduate

Studies, University of South Africa, South Africa.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
312
THE FLIPPED CLASSROOM: MEDIA HYPE
OR EMPIRICALLY BASED EFFECTIVENESS?
Lars-Frederik Weiß, Gunnar Friege
Leibniz University Hannover, Germany
E-mail: weiss@idmp.uni-hannover.de, friege@idmp.uni-hannover.de
Abstract
Few teaching concepts have received as much attention in recent years as the Flipped Classroom (FC).
This is clearly linked to media attention on the concept, which is currently being intensified by the Corona
pandemic. In this research, the media rise of the concept is explained and the discrepancy between
popularity and empirically validated research results is clarified. While the literature basis for the FC
has grown enormously over the last decade, reviews and initial meta-analyses have shown, through their
filter criteria, that not many of studies can meet scientific standards. The comparison of individual studies
is particularly difficult, because there is no uniformly accepted framework and no uniform definition of
the concept. For this reason, one of the goals of this research was to give an overview of the variety of FC
definitions, discuss its most controversial characteristics, and to provide the key elements for a definition
of the FC. A comparison of selected studies and reviews shows an assessment of the effectiveness of the
teaching concept. The findings shed light on potential problems and challenges, and provide directions
for future research.
Keywords: active learning, digitalization of education, e-learning, flipped classroom
Introduction
The Flipped Classroom (FC) method, as a vehicle for the digitalization of education, has
become a focus of digital teaching methods. Traditional learning by means of books and notes
can be supplemented and supported by digital media, such as learning videos and simulations,
and this is especially true for the Flipped Classroom concept. "Learning is increasingly taking
place virtually, whether as e-lecture, free online course, in an inverted classroom or through
learning apps" (Schmid et al., 2017, p. 5). Universities see the digitalization and implementation
of "innovative" didactic concepts as solutions for problems such as heterogeneous student
populations and individualized learning (Zervakis & Mooraj, 2014). With an increasing
digitalized society, teaching at schools and universities is also becoming more and more
digital through new didactic concepts such as the Flipped Classroom (Schmid et al., 2017,
p. 50), not to mention the worldwide increase of digital learning material and methodologies
in home-schooling settings and online university courses due to the Covid-19 pandemic (see
Figure 1 for an increase in search interest after the recent Covid-19 outbreak). "The Flipped
Classroom method (Inverted Classroom or, much less frequently, reversed teaching) has
been very much present in public and scientific discussion about modern forms of teaching,
including the use of digital media, for some years now" (Werner et al. 2018, p.13). Despite
its increasing popularity and media attention, the effectiveness of the Flipped Classroom as a
teaching concept remains debatable. In addition to this, research on the FC is still in its early
stages. The present study is intended to provide an overview of the current state of research.
The development and definition of the FC is discussed in detail and the related media hype
is highlighted. The FC is realized in universities and in school teaching, i.e., under different
organizational and curricular conditions. The differences between realizations of the FC in
schools and universities is explicitly discussed. An analysis and comparison of meta-analyses

Lars-Frederik WEIß, Gunnar FRIEGE. The flipped classroom: Media hype or empirically based effectiveness?
PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
and reviews, as well as frequently cited peer-reviewed articles, provide the basis for drawing 313
implications and conclusions for further research, teaching, and educational policies. Previous
misconceptions and misunderstandings of the concepts are addressed.
Development of the Flipped Classroom – a Media Hype?
Media interest in the FC has been growing worldwide since 2012, and can be easily
visualized using, for example, Google Trends (Fig.1). Google Trends is only used here to show
public interest in the FC, not as a scientific method of analysis. The context in which the term
was searched for remains unknown.
Figure 1
Temporal Interest in the Search Term "Flipped Classroom
Note: created with Google Trends (www.google.com/trends), status: 26.02.2021

(value 100 indicating the highest popularity)
In line with the strong growth in FC media interest, it is increasingly becoming the
focus of research. “The flipped classroom is a phenomenon that is spread rapidly at schools
and universities around the world, also attracting a growing body of the research” (Zupanec et
al., 2018, p.136). A more scientific picture emerges from the analysis of publication figures on
the subject of FC. The Web of Science lists 5292 publications on the Flipped Classroom for
the period 2000-2020. The strong increase in published articles from around 2012 is striking
(Table 1).
Table 1
Number of Articles Published per Year
Publication Years Number of Records

2020 844
2019 964
2018 961
2017 997
2016 719
2015 553
2014 268
2013 90
2012 18

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
314 Interest in the FC is substantially impacted by video portals and learning platforms,
whose users multiplied from 2010 onwards. In general, there is a clear dominance of North
American literature on the topic of FC. "While much of the U.S. dominance in publications could
be accounted for by a general Anglo-American dominance in academic publishing, it should
also be noted that the flipped classroom approach was first popularized in secondary education
by two teachers, Bergmann and Sams, both of whom are active within the U.S. educational
context" (Lundin et al., 2018, p. 9). A search of the Web of Science literature database confirms
this, with over 1300 publications from the U.S. (https://wcs.webofknowledge.com/). The large
number of newly published articles is partly due to the fact that the implementation of and
research on the FC is largely based on the initiative of single lecturers and teachers. As a result,
many publications are experience reports and recommendations for action, but rarely structured
empirical studies with hard evidence for teaching and learning. First meta-analyses confirm
the limited study situation of empirically valid research. Wagner et al. (2020) found only 33
empirical studies from 1054 on the FC that met scientific standards (period Jan. 2012 - Jan
2018). The large number of studies that have been conducted does not allow a generalizable
conclusion to be drawn about the effectiveness of FC, as there is a lack of information about the
control groups and the design of conventional instructions.
The concept began in 1997 with Eric Mazur's “Peer-Instruction". Walvoord and
Anderson published "Effective Grading” in 1998, and the article "Inverting the Classroom:
A Gateway to Creating an inclusive Learning Environment" was published in 2000 by Lage,
Platt and Treglia. Broad research on the concept started 15 years later. While Mazur (1997),
Walvoord and Anderson (1998) and Lage et al. (2000) focused only on university teaching, the
implementation of the FC in schools started in 2007 with Bergman and Sams (in the Anglo-
American sphere) (Bergmann & Sams, 2012). While the Inverted Classroom has its origins in
American universities, the FC was initially established at schools. Consequently, both terms are
frequently used in literature. However, the FC is the most common term for referring to concepts
used in schools and universities. Regardless of the taxonomy of the concept, the rapid increase
in media and research interest is striking, though as the FC is often thought of as part of the
digitalization of education, this is not surprising. The boom in free video and teaching material
portals (such as YouTube or Khan Academy) and the availability of digital playback devices
(especially smartphones) have been making significant contributions to the popularization of
the concept. According to the Jugend, Information, (Multi-) Media (JIM) study (2018), which
examined the media usage of young people between the ages of 12 and 19, 99% of German
households own smartphones, 98% own a computer/laptop, and 98% have internet access.
These results are most likely comparable to other industrialized countries. However, simply
watching instructional videos does not automatically mean that something fits the FC concept.
Nevertheless, easy access to a large number of such videos will promote the popularity of the
concept.
The FC is also being discussed intensively outside of scientific publications, using striking
headlines, strong terms and claims of successful teaching (for example: teaching innovation).
The German daily newspaper Süddeutsche Zeitung reported that: "Professor Bonnet abolished
his lectures. Since then, his students have been performing better in exams." One of the
most popular weekly magazines, Der Spiegel, states: "Cologne professor is a YouTube star",
“Cologne professor of mechanical engineering uploads his lectures on YouTube. Since doing
so, more people have been passing their exams." However, the daily Frankfurter Allgemeine
Zeitung speaks of an "[...] online attack on teaching". The New York Times proclaims “The
virtual teacher has arrived […] turning traditional education on its head.” In addition to reports
in various newspapers, the personal experiences of teachers, consultants and FC coaches have
been published in various internet blogs and magazines, which offer recommendations for
action and often provide free video material. For example, the human resource magazine Best of

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
HR-Berufsbilder reports: "How the Flipped Classroom improves learning: teaching and digital 315
media belong together.” The FC teaching innovation and similar or synonymous concepts (like
Inverted Classroom or reversed teaching) are of increasing interest, and obviously associated
with high expectations for scientific communities, teaching settings and in public. Although
teaching settings are quite different, particularly for schools and universities, there is still a
striking lack of differentiated views on these highly diverse settings.
Clarification of the FC Concept
Even if the basic concept seems easy to understand at first - outsourcing the traditional
lecture content to a self-learning phase and working on cognitively demanding tasks in a
presence phase instead of as homework - a look at the literature reveals a lack of clear distinction
between the FC approach and traditional teaching (O'Flaherty & Phillips, 2015; Tucker, 2012).
As O'Flaherty & Phillips (2015) and Tucker (2012) state, definitions of the FC concept
differ greatly. "While published research at times refers to teaching and learning approaches
which could potentially be classified as FC approaches, they rarely label it as such" (Abeysekera
& Dawson, 2014, p. 5). For this reason, an overview of the most common definitions was created
in chronological order, providing a simple overview (see Table 2). The selected definitions are
taken from review articles or peer reviewed, frequently cited articles.
Table 2
Definitions of the Flipped Classroom Concept
Definition Reference
“An inverted (or flipped) classroom is a specific type of blended learning design that uses
technology to move lectures outside the classroom and uses learning activities to move Strayer (2012, p.171)
practice with concepts inside the classroom.”
“Recording in-class activities to convey a course: Students watch the video before the class
and use the class time to solve complex concepts, answer questions, and students are Stone (2012, p. 405)
encouraged to learn actively as well as create bonds with daily lives.”
“We define the flipped classroom as an educational technique that consists of two parts:
Bishop & Verleger
interactive group learning activities inside the classroom, and direct computer-based
(2013, p.9)
individual instruction outside the classroom.”
“pedagogical approaches that:
1. move most information-transmission teaching out of class;
Abeysekera & Dawson
2. use class time for learning activities that are active and social; and
(2014, p.4)
3. require students to complete pre- and/or post-class activities to fully benefit from in class
work.”
Association of Flipped
“Flipped Learning is a pedagogical approach in which direct instruction moves from the group
Learning Network
learning space to the individual learning space, and the resulting group space is transformed
(2014, p.1);
into a dynamic, interactive learning environment where the educator guides students as they
Bergmann & Sams
apply concepts and engage creatively in the subject matter.”
(2014, p.14)
“In short, the flipped classroom is a pedagogical approach which moves the learning
contents taught by teachers’ direct instruction to the time before class in order to increase the
Hwang et al. (2015,
chances for the students and teacher to interact. Therefore, teachers would have more time
p.452)
to guide the learning activities and solve students’ problems in order to promote the learning
effects.”
“Flipped Learning is a framework that enables educators to reach every student. The Flipped
approach inverts the traditional classroom model by introducing course concepts before
ALAAS (2018)
class, allowing educators to use class time to guide each student through active, practical,
innovative applications of the course principles.”

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
316 While all the definitions suggest that knowledge transfer should be separated from the actual
classroom into self-learning phases, and that the time which has now been freed up should
be used for active learning, opinions are divided on the use of media for knowledge transfer:
"Although some definitions of the flipped classroom approach require video lectures [...], others
provide a broader approach" (Wood et al., 2016, p. 4). Bishop and Verleger (2013) exclude
articles in their review that do not use video as a medium for imparting knowledge during
the self-learning phase and instead make use of other media, emphasizing how videos are a
dominant component of the FC. "We restrict this definition by excluding designs that do not
employ videos as an outside of the classroom activity. While a broad conception of the FC may
be useful, definitions that become too broad suggest that assigning reading outside of class and
having discussions in class constitutes the flipped classroom" (Bishop & Verleger, 2013, p. 5).
They legitimize the restrictions of their definition, among other things, by learners rarely reading
texts conscientiously. This means that the effectiveness of reading tasks for preparation would
not reflect their FC definition (see Table 2). However, it should be noted that the absorption of
information from videos would often be superficial or incidental. "Thus, self-study in the non-
school classroom runs the risk of being ineffective" (Fischer & Spannagel, 2012, p. 238). The
digitalization of knowledge transfer through videos could therefore lead to teachers focusing
on the selection and creation of videos - which should certainly be done to a certain extent - but
neglecting the actual planning of classroom teaching, with its activating elements.
“Use of video as the prime means of direct instruction has resulted in an emphasis on the
effectiveness of video production and delivery, with the teacher or school selecting videos
rather than considering the broader range of direct teaching methods in a traditional classroom.”
(Sharples, 2014, p. 16)
Werner et al. (2018) also point out that learners need guidance in working with videos
as learning tools, ensuring an active and sustainable use of the material provided. They further
state that "[...] the fact that children and young people use digital devices on a daily basis
in their leisure time does not mean that they use digital media in learning contexts without
problems" (Werner et al., 2018, p. 61).
At this point, it does not seem conclusive to delimit the FC primarily by means of the
medium used during the self-learning phase. Such a definition excludes all teaching approaches
which do not use videos but in principle correspond to the core FC concept. Consequently,
there are definitions that are restrictive to varying degrees. Some definitions define the FC
through technology and video lectures, and thus propagate the FC as new and innovative. On
the other hand, there are more open definitions that allow teaching concepts that may have been
practiced for several decades to be considered FC. According to Strayer (2012), the FC belongs
to the super-category of e-learning, i.e. electronically supported learning. It can be seen as a
special form of blended learning. Unlike traditional blended learning, conceptual knowledge
and facts are almost exclusively learned in a self-learning phase, which precedes the application
of what has been learned in the presence phase in an FC setting. In traditional blended learning,
knowledge transfer takes place both through e-learning and in traditional lectures (O'Flaherty &
Phillips, 2015; Baumgartner, 2011). Unlike Strayer (2012), both Ozdamli and Asiksoy (2016)
and He (2016) stressed that the FC was not to be understood as simply watching online videos.
The focus is much more on interactive activities during lessons and face-to-face time with
teachers. It is noticeable that early definitions regard the FC as e-learning (Bishop & Verleger,
2013; Stone, 2012; Strayer, 2012) while later ones (Abeysekera & Dawson, 2014) have
expanded upon this, making e-learning no longer a necessary prerequisite for the FC concept.
In an attempt to summarize the various definitions into an overview of core constituents, Lo
(2018) has assumed the following elements to be essential for the FC:

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
1. Out-of-class, individualized, computer-based instruction, focusing on information- 317

transmission teaching.
2. In-class, interactive group learning activities that emphasize peer-assisted learning
and problem solving.
In accordance with Lo (2018), Finkenberg (2018) visualizes a basic form of the FC, which
is to be understood as the "lowest common denominator" of various forms of implementation.
The presence phase is divided into assessment and application.
Figure 2
Adapted Basic Form of the Flipped Classroom, in the Style of Finkenberg (2018)
Self study phase Presence phase
Instruction Assessment Application
work on preparatory material and Quiz with personal Partner/Group work with
learn key concepts, ask questions if response system: tasks in personal responsibility with mutual
needed concept questions about assistance, “active learning”
the content of the
learning materials
While Lo (2018) has focused on computer-based instruction, a more open approach like
Finkenberg's (2018) - though restricted by an assessment phase – seems to be the most ap-
propriate. The concept of the FC should not be defined by the supporting medium, but by the
phasing of teaching and learner activation, as well as by the role shift of teachers and learners
away from traditional teaching. The FC concept can thus be seen as a synthesis of various peda-
gogical approaches that shift the transfer of knowledge from classroom to self-learning phases,
shape classroom time through active learning and require students to work independently or in
a self-regulated manner (Abeysekera & Dawson, 2014; Al-Zahrani, 2015; Strayer, 2012).
Based on the discussion and theory of FC, the following definition is proposed: The
Flipped Classroom is a form of teaching in which students prepare for the actual lesson
asynchronously and independently in a self-study phase, using materials provided by the
teacher. The self-learning phase is supplemented by a subsequent presence phase, in which
the acquired knowledge is examined, applied and developed in a student-centered manner.
The teacher acts as a learning guide, who creates the learning environment for the successful
consolidation of knowledge.
Differences between the FC and Conventional Teaching
“A judgement about the learning effectiveness of the Flipped Classroom must be based
on the lessons it is compared to. The term "traditional teaching" is diffuse and in many
studies it refers to different forms of teaching - from the classical lecture without any
activity of the learners to question-developing lessons that can be designed with a high
degree of student activity.” (Finkenberg, 2018, p. 24)
In the FC literature, traditional teaching (if described at all) is primarily understood as

the passive reception of knowledge during a teaching period (cf. Bishop & Verleger, 2013).
The passive reception of knowledge describes a frequently held idea about learning. Pupils
"[...] generally see learning as the simple transfer and storage of knowledge [...]. Their learning
behavior in class is correspondingly 'passive'" (Kircher, 2015). Likewise, a similar manner of
thinking can be observed in many teachers: "[...] the view that knowledge can be passed on

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
318 (transported to the student) dominates" (Kircher, 2015, p. 662). Furthermore, the independent
processing of exercises outside of class, mostly in the form of homework, is a typical component
in teaching defined as traditional. The same applies to teaching at universities. In such settings,
traditional teaching is understood as the passive acquisition of knowledge within the framework
of lectures. Exercise groups, in which tasks that were typically worked on at home before being
checked in class, are another passive element of traditional university teaching, especially in
physics and mathematics.
The role of the teacher also differs between the FC approach and teaching with other
concepts that are not structured according to the basic FC concept (see Figure 2). This means
that they do not distinguish between an asynchronous self-learning phase and a synchronous
presence phase. While a fear of the FC making the physical presence of teachers superficial
exists – since students would eventually acquire the knowledge without the presence of a
teacher – this fear is countered by a shift in the role of the teacher during a presence phase of
the FC concept.
“In a traditional teaching model, the instructor facilitates content attainment through various
means in a classroom setting. Students are then given the responsibility of applying the concepts,
generally in the form of homework assignments. In a flipped model, the roles are reversed, with
students being responsible for attaining the content before coming to class, at which time the
instructor facilitates the application process. It appears that the main difference between these
models is the role of the instructor.” (Jensen et al., 2015, p. 2)
The role of the lecturer is changing dramatically, especially in the university environment.
While the conventional role of the lecturer is characterized mainly by lectures with little student
activity, the presence phase of the FC now demands a more complex didactic embedding of the
teaching content, in which the focus is on student activity. Content knowledge and the knowledge
of methodology and didactics are required to facilitate a suitable learning environment. The
transfer of knowledge and its application and consolidation now play a crucial role for the
lecturer. Intensive and broad preparation is also necessary on the part of the lecturers, as the
presence phase invites students to ask questions, even beyond the core content of the individual
subject areas.
A change in roles can also be observed in school contexts, albeit less drastically than
in university settings. In the FC concept, the teacher is responsible for providing materials
that allow students to prepare for the actual lesson in a self-regulated manner, since the mere
transfer of knowledge is outsourced from the classroom (Lankford, 2013). As before, the
teacher must design a learning environment in which knowledge can be applied and developed,
and on a more individual basis in the FC. The activities of students and the use of cooperative
forms of learning require the teacher to carefully examine the learning group and the learning
situation. Here, the importance of the teacher in the FC concept again becomes clear. “This
may be contrary to the belief of many that the teacher becomes less important when introducing
videos as a lecturing component” (Fredriksen, 2020, p. 393). The teacher accompanies the
learners as they work with more complex tasks, while the students carry out the less cognitively
demanding acquisition of the largely declarative knowledge in a self-organized, individual, and
asynchronous manner (Gilboy et al., 2015).
Not only are the roles of university lecturers and teachers changing, but also those of
students and learners. While university students should already be well-trained to acquire
knowledge in a self-regulated fashion and to adapt it to their individual learning behavior, this
poses a far greater challenge for students — again, it becomes clear how central the role of the
teacher in the selection and provision of materials is. In the presence phases, students are now
confronted with cooperative forms of learning and a good number of activities. In the STEM

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
subjects, traditional teaching instead relies on the independent calculation of exercises without 319
didactic and methodological embedding. This can make working with fellow students in group
work and different and new student-centered methods a challenge. Furthermore, the modular
and more flexible structure of courses can also lead to difficulties in organizing learning.
As can be seen from the overview of definitions (Table 2), the definition of the seemingly
simple structure of the FC is anything but trivial. A change to a more open definition, where an FC
concept is possible without an e-learning component, is considered reasonable. A differentiation
to conventional teaching is difficult to make, because conventional teaching varies greatly
between teachers, lecturers, schools and universities. Nevertheless, a differentiation through
role shifts of teachers and students seems to be a good and concise distinguishing feature.
There have already been some meta-analyses and reviews on the topic of FC, with
different focuses following strict guidelines (such as PRISMA). Instead of a further meta-
analysis according to these guidelines, existing meta-analyses and reviews will be used to form
the foundation of the present literature review. As mentioned earlier, only a few articles have
met scientific standards, since most of the extensive literature on FC is based on individual
reports and recommendations for action. In this study, the reviews and meta-analyses were
used to contrast frequently cited studies. Field reports, recommendations for action and non-
peer-reviewed articles were not included in the discussion of the current state of research.
The citation index, impact factors and cross referencing were consulted, in order to identify
frequently cited and popular studies that have significantly shaped the state of research and
opinions on FC. The Web of Science and Google Scholar were mostly used to screen and filter
studies. As a prerequisite for inclusion in the research findings (Table 3), studies must compare
the FC approach to a non-FC approach and must have been conducted in a STEM subject. The
26 articles listed were read and analyzed in detail. Information about the test instruments, the
subject area of the study, the constructs examined and the effect sizes of significant results were
highlighted.
Research Results
Apart from a contradictory definitions of the FC, previous research has also provided
mixed results and has largely been limited to research based on the initiative of individual
lecturers, using questionnaires to evaluate students' attitudes towards the new concept or to
compare it to the traditional teaching format (Blair et al., 2016; Mason et al., 2013; Newmann
et al., 2016; Strayer, 2012; Teo et al., 2014; Thai et al., 2017). Research on the FC is almost
exclusively limited to university teaching, with research in the Anglo-American sphere
dominating. Despite an enormous growth in literature (especially in the last two years and
even outside the USA), "the Flipped Classroom still has exotic status in the German university
system, but especially in the German school system" (Finkenberg, 2018, p.40). According
to this, didactic research on the FC is the exception in the German (and, most probably, the
whole European) language area. One such example is "Flip your Class!”, which examines the
implementation of the FC at three schools in Berlin, and another is the "Flipped Classroom"
project, with 28 pilot schools in the state of Schleswig-Holstein. The studies suggest that the
integration of digital videos in the classroom is suitable for learning with students. However,
the study’s approaches do not represent an FC in accordance with the definitions presented. The
evaluation of the "Flipped Classroom” project is limited to written feedback from individual
schools, which report on the increased motivation of learners (Ministry of Education and
Vocational Training, 2017). The accompanying research is to include a survey of learners and

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
320 teachers, but a more detailed evaluation of the project has not yet been published. The results
of the data collection by means of expert interviews, student observations and surveys of the
"Flip your Class!" in Werner et al. (2018) must be critically questioned. Especially since the
implementations do not correspond to the common definitions of the FC, no conclusions should
be drawn about its effect at this point. Furthermore, the data collection and evaluation in the
Flipped Classroom project is not transparent. A further investigation of the FC in schools was
conducted by Finkenberg (2018). A study examining the FC in the physics lessons of an upper
secondary school of 151 students found a significantly better learning performance in the FC
group, with medium strength effects (see Table 3). Furthermore, a significant increase in the
physics-related self-concept in the treatment group (FC) was observed. Finkenberg's study can
be seen as evidence of the effectiveness of the FC in school physics lessons.
A large number of previous studies have been conducted in the STEM field. Because
the processing of authentic, practice-oriented problems requires knowledge of a multitude of
laws and concepts which must be mastered before they can be applied to such problems, "many
STEM subjects contain an abundance of principles and (seemingly) abstract concepts which
students need to 'know' before being able to move on to more practical, authentic applications"
(Huber & Werner, 2016, p.267). These concepts and laws, which form a basis for practical
applications and problems, could be transferred particularly well into the self-learning phases
of the FC concept.
In the following, an in-depth overview of publications from the STEM sector will be given.
Only studies that have been published in peer-reviewed sources that meet scientific standards
and were published after 2012 are listed. Furthermore, the selected articles meet the Flipped
Learning Network’s definition of the FC (Table 2). For this reason, conference proceedings and
conference contributions were not included in the overview. A review of previously published
studies - also from the field of medicine - can be found in the review by Bishop and Verleger
(2013), which is discussed later. In addition to the constructs examined, the overview also lists
the effect size and any significant results. In some cases, no effect sizes were published. In such
cases they were calculated retrospectively, provided that sufficient parameters were available
for the calculation and underlined in Table 3. The test instrument used for data collection was
also listed. Test results from self-developed tests were compared. These are marked as "self-
developed". All the studies compared the FC approach to conventional teaching and were
classed as effective, making no difference, or less effective. The vast majority of these studies
examined the learning gains after implementing the FC approach. Few of the studies examined
any other affective constructs of the study participants.
Table 3
Findings of Peer-Reviewed Studies Comparing the Flipped Classroom and Traditional
Teaching in the STEM-Field (*marks statistical significance, __ marks calculation by authors)
Findings Source IC ES Field of Teaching Test Instrument

Pre-Post surveys,
Amresh, Carberry University Engineering
FC more Learning gain d=1.12 Exam Scores (self-
&Femiani (2013) (introductory
effektive Computing self-efficacy* developed)
programming course)
University Engineering
Mason, Shuman, Learning gain* (only on Exam scores
NA (mechanical
and Cook (2013) certain items) (self-developed)
engineering)
McGivney‐
Student’s perception on University Physics Surveys and
Burelle and Xue NA
teaching (calculus) Interviews
(2013)

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Learning gain* University Statistics Exam scores 321

Wilson (2013) d=.57
course (self-developed)
Student’s perception University Math
Talbert (2014) NA Case Studies
(>70% preferred FC) (Linear Algebra I)
Engagement* d=1.05
Perception surveys
Beapler, Walker Flexibility* d=.38
University (Whiteside et al
& Confidence* d=.78
Chemistry 2010; Walker et al
Driessen (2014) Learning outcomes* d=.38
2011)
Effective use* d=.41
Learning gain*
Chao, Chen & learning attitudes, High School (11th grade Pre-Post Tests
d=.79
Chuang (2015) motivation and self- Engineering) (self-developed)
evaluation
interest in cooperative College and
Chiang & Wang learning, engagement, University
University
(2015) self-directed, self- NA Classroom
Engineering
regulating, and self- Environment
determined skills Inventory (CUCEI)
University Math Exam Scores and
Wassermann et Learning gains* ηp=.04
(Calculus III) survey
al. (2015) In-class communication* d=1.39
(self-developed)
Physics
Atwa, Rosseni & Learning gains* High-School Physics Achievement
d=.47
Hussin (2016) (11th grade) Test (PAT) (self-
developed)
Learning gains*, Exam scores
Sun & Wu (2016) ηp=.06 University
Student-teacher Survey (Sun, Shih
Physics
interactions & Wang (2007)
Exam scores
University (self-developed)
Şengel (2016) Learning gains* d=0.57
Physics Exam scores
(self-developed
University Math Exam scores
Heuett (2017) Learning gains* NA
(non-math majors) (self-developed)
Adams & Dove University Math
Learning gains*, Pre-Post Tests
(2017) r=.5 (Calculus and Analytic
Perception of learning (self-developed)
Geometry I)
ηp=.06
Finkenberg
Learning gains* ηp=.03 High School Physics Pisa-2015 Items
(2018)
Self-concept in Physics* (11th grade) (ST130)
Eldy, Elnetthra
Learning gains* Pre-University Pre-Post Tests
(2019) NA
(Mechanics and (self-developed)
Thermal Physics)
Tolks et al. (2016) Learning gains* ηp=.06 Pre-Post Tests
University
(self-developed)
Physics
Pre-Post Tests
Wei et al. (2020) Learning performance* NA Middle School Math
(self-developed)
Mason, Shuman, University Engineering
No Student’s perception of
and Cook (2013) NA (mechanical Surveys
difference teaching
engineering)

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
322 McGivney‐
University Exam scores
Burelle and Xue Learning gains NA
Physics (calculus) (self-developed)
(2013)
Learning gains, college-level information Pre-Post Tests
Davis et al.
Student’s perception of NA systems spreadsheet Surveys
(2013)
the course course (self-developed)
MC examination
Baepler, Walker
University (GC08C by ASC)
& Driessen Learning gains NA
Chemistry and exam (self-
(2014)
developed)
University Engineering
Velegol et al. Learning gains Pre-Post Tests
NA (environmental
(2015) Student’s metacognition MSLQ Survey
Engineering)
EMCS (Singh &
Yong et al. (2015) Learning gains NA University Math Rosengrant 2003)
Exam scores
Limueco & High School Teaching
Learning gains NA (self-developed)
Prudente (2019) (9th grade)
University
FC less Gundlach et al. Exam scores (self-
Learning gains NA Statistical
effective (2015) developed)
Literacy Class
Note. FC = Flipped Classroom, ES = effect size, IC = investigated construct
Even if the overview classifies a large number of studies as more effective, the
significance of the individual studies remains limited. The design of the FC varies greatly
between individual studies. Furthermore, the definition of the previous conventional teaching
remains fairly unclear. As the discussion of the various attempts at defining the FC has already
shown, the concept can be used in a wide variety of ways. Therefore, a comparison of the
effectiveness only seems to make sense if the framework conditions are transparent and easily
comprehensible. In their review, Bishop and Verleger (2013) criticized the unclear framework
conditions, especially with regard to the didactic design of the attendance phases. 24 published
articles met the requirements of the restrictive definitions of Bishop and Verleger (2018), while
eleven studies did not meet these requirements. They state that perception of the concept tends
to be positive in the reviews studied, but one group of students described the FC approach very
negatively: "Despite differences among studies, general reports of student perceptions were
relatively consistent. Opinions tended to be positive, but there were invariably a few students
who strongly disliked the change" (Bishop & Verleger, 2018).
The "Monitor Digitale Bildung. Die Hochschulen im digitalen Zeitalter" (2017) showed
that 64% of students in Germany already used videos for private learning. “Beyond Millennials:
The Next Generation of Learners” (2018) showed that 59% of students aged 14-23 voted
YouTube as their preferred medium of learning, ranked it more highly than in-person group
work. At this point, it has to be again made clear that videos can differ greatly. The quality and
nature of videos varies as much as the implementation and design of the attendance phases.
Two of the studies evaluated the performance of the students and found significantly higher
scores in exam questions related to the material conveyed in the videos (Moravec et al. 2010).
Day and Foley found that students scored significantly better on homework, tests and projects.
However, neither conventional teaching nor the implementation of the FC are discussed in
detail. The assumption that the provision of video recordings would mean students would no
longer participate in classroom events and that direct exchange with the teacher would become
superfluous, often expressed in the context of the FC, cannot be confirmed (Stone 2012). More

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
precise results are not given. However, it is also stated that students prefer "live in-person 323
lectures" to video lectures. This is a result which is not discussed further in the Bishop and
Verleger review (2013), but at first is very surprising.
The very limited number of studies conducted in a school setting is also striking. Only
five of the 26 studies listed examined the FC in a school setting, though most of them were
able to demonstrate significant improvements in learning groups. Even after the publication of
Bishop and Verleger (2013), the focus of FC research was centered on university environments
rather than schools (see Table 2).
Even if the comparability between the studies is only guaranteed to a limited extent,
some advantages and disadvantages of the FC can be determined from the practical examples.
Karabulut‐Ilgu et al. (2017) mentioned flexibility as the most frequent advantage in a large
number of studies (e.g. Buechler et al., 2014; Kiat & Kwot, 2014; Mok, 2014; Simpson et al.,
2003; Velegol et al., 2015). Particularly from the perspective of the learners, the permanent
access to instructional materials enables flexible design in the self-learning phase. This allows
for an individual phase of knowledge acquisition adapted to the learner's own learning type.
This also includes the possibility of varying the playback speed of videos or repeating individual
passages. Videos often act as substitutes for reading. The flexible outsourcing of knowledge
transfer can generate scope for working on more complex tasks during the attendance phases
(O'Flaherty & Phillips, 2015), allowing for more peer interaction time. Learners particularly
appreciated working with their peers and the role of the teacher as coach or learning guide.
Karabulut-Ilgu et al. (2017) also point out that some studies argue in favor of the FC, as this
approach promotes learners in skills such as life-long-learning (Luster-Teasley et al., 2014),
learner autonomy (Kim et al., 2014; Mok, 2014), critical thinking (Chetcuti et al., 2014) and
interpersonal skills (Yelamarthi et al., 2015). Another advantage of the FC approach is the
commitment of the learners. In a number of studies (see e.g. Karabulut-Ilgu et al., 2017), it has
been observed that learners are better prepared for classroom instruction (Chetcuti et al., 2014;
Jungic et al., 2015; Mok, 2014; Popadopoulos & Roman, 2010) than they are with conventional
teaching approaches. Werner et al. (2018) found that the FC offers more opportunities for
differentiation, and that students allocate their time much more effectively. "The former
was implemented, among other things, through tasks with different levels of requirements,
competence checks, interview results and additional tasks for faster or more capable children
and adolescents with different roles (e.g. listener and assistant) in group work" (Werner et al.,
2018, p. 58-59).
Despite the many potential advantages that are strongly dependent on the individual
implementation of the FC concept, it should not be overlooked that a significantly higher
workload for teachers, especially before and during the lessons, is sometimes seen as the
greatest challenge (Clark et al., 2014; Ghadiri et al., 2014; Kalavally et al., 2014). This is due
to the increased time required for the preparation and selection of materials for the self-study
phase, and also for the individual support of each learner during the presence phase and the
methodological and didactic design of the latter. While there is already an abundance of learning
videos from third party providers at school level, videos for special topics at a university level
often have to be produced by the lecturer. However, if videos that were produced for the self-
study phase can be used again in subsequent years, the time required for preparation on the part
of the teachers and lecturers is considerably reduced. Learners often complain about technical
problems, such as internet connection speeds (Everett et al., 2014). Observations have shown
that technical problems often occur due to a lack of equipment or non-functioning school servers
(Werner et al., 2018). Furthermore, it becomes clear that learners are often overwhelmed by the
higher responsibility associated with self-organized learning (Margoniner, 2014; Werner et al.,
2018). Therefore, a comprehensive introduction to the method is particularly important. An
introduction to the teaching methods should make the requirements and the process transparent

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
324 to learners. Learning with videos, if they are to be used, should also be something for which
training is done in advance. This problem can be applied to universities and schools alike.
For example, self-regulated learning is less of a problem in universities than in schools, but
the break with conventional teaching - especially in advanced semesters - often brings about
new problems. As explained in the role shift, lecturers at universities also need to be trained to
use the FC in an active way during the presence phases. The most significant advantages and
disadvantages from the research are shown in Table 4.
Table 4
Most Prominent Advantages and Disadvantages in the Flipped Classroom Literature
Advantages Disadvantages
Learners can perform better (see Table 3) Success of the method depends heavily on learner‘s
Learners can learn at their own pace (e.g. watch videos preparation during the self-study phase (Karanicolas et al.,
several times) (Nouri, 2016) 2016)
Learners work on cognitively more demanding tasks while Mandatory internet connection and playback media
being supervised (Lee & Lai, 2017) (Clemens et al 2013; Everett et al., 2014)
Teachers are more flexible in class (Bergmann & Sams, Learners are often overwhelmed with new approaches
2014) (Amresh et al. 2013; Bland, 2016)
Mostly positive perception of the FC by learners (Bishop & Learners have problems organizing themselves within the
Verleger, 2013) FC (Margoniner, 2014)
Video material can be used several times Increased workload, especially on forehand (Ghadiri et al.,
2014; Kalavally et al., 2014)
Insights gained from research into the FC approach make it clear that a theoretical
framework for a uniform design and implementation is still lacking. The basic form of the
Flipped Classroom, based on Finkenberg (2018), offers one possibility for unifying the design.
Subject didactics are only involved in the planning, implementation and evaluation of the FC,
both at the university level and in a school context, in exceptional cases. This is particularly
evident in the didactic design of the self-learning and presence phases. Learning objectives,
methods chosen and material used remain unmentioned in most studies.
However, transparency regarding the design of the attendance phases is a prerequisite
for being able to make statements about the effectiveness of the FC, since the success of the
concept, like the success of traditional teaching, depends largely on lesson planning (Lee &
Lai, 2017). Freeman et al. (2014) have already shown that learners perform better in the natural
sciences if they are involved in various active learning tasks, i.e. problem solving in groups,
personal response systems and workshops.
The positive results for the effectiveness of the FC when compared to conventional teaching
from the listed studies may be due to the "active learning" of the students. This hypothesis was
tested by Jensen et al. (2015) in a study comparing the FC approach to traditional but active
learning university teaching. "Results show that the benefits of the flipped model over a non
flipped active-learning model are insignificant [...] when holding all other variables constant"
(Jensen et al., 2015, p.10). In their conclusion, Jensen et al. (2015) stressed that the additional
expenditure for the implementation of a FC would not provide significant added value if the
students were to participate in lessons which were already designed through active learning.
Nevertheless, apart from missing evidence to support this claim further, the FC concept is a
possibility for putting the active learning of students into the focus of the learning activities.
Young et al. (2015) came to a similar conclusion. They compared the performance of students
after one semester using pre- and post-tests in a control group design for an introductory course
to differential calculus, and found no significant differences between the groups. They also
pointed out that the general conditions and the method of implementation are crucial for the

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
success of the Flipped Classroom. "Therefore, it is possible that as we improve our instructional 325
methods in the flipped classes, we may see statistically significant differences in student learning
and attitudes." (Yong & Lape, 2015, p. 919) Furthermore, they point out that more research is
needed to determine under which conditions and in which contexts the FC approach will yield
the best results for as many learners as possible.
The different ways of implementing the FC approach, or rather the lack of a common
framework for its use, quickly became the focus of AALAS— an association of 100 teachers and
lecturers that developed a framework for the FC— which founded a "Global Standards Project"
to counteract the problem (https://aalasinternational.org). "The Global Standards Project offers
a framework and benchmarks to enable Flipped Learning practitioners to access and employ the
most current international best practices" (AALAS, 2018). The result of the project is reflected
in 187 "best practices", including "understand each student's cognitive needs", "promote
collaborative and group work", "explain to students how they can become effective Flipped
Leaners". According to this, the goal of the Global Standard Project is to create global standards
for teaching according to the FC approach. To what extent 187 very general "best practices"
offer a framework for implementing the FC or embody much more general recommendations
for teaching remains questionable. Many of the best practices can be used for both flipped
and traditional lessons. The important distinction between traditional teaching and FC remains
blurred. Furthermore, the best practices do not standardize the implementation of the approach.
187 different recommendations do not allow a clear, guided and structured implementation,
but rather apply to all possible forms of implementation, regardless of the chosen teaching
methodology.
In summary, it can be said that there is some evidence to support the effectiveness of
the FC (see Table 3). A significant amount of research has been carried out, especially in the
university sector. Research in the school sector is still very limited. Nevertheless, there is also
some positive evidence. While many studies examine the learning gains of students, empirical
scientific research of affective characteristics of learners (beyond reporting impressions and
experiences such as motivation and interest) is lacking.
Research in Schools and Universities
While research has progressed in recent years, it should be emphasized how FC

approaches differ between universities and schools. Results from more advanced research
in university teaching cannot be transferred to the school context. Compared to university
teaching, its implementation in schools requires different framework conditions, starting with
the timing of lessons and moving on to the ability to organize learning independently. The
structural framework between universities and schools change, as well as the potential problem
areas. There is a tendency for most learners to evaluate university teaching based on the FC
concept more positively than conventional teaching. Nevertheless, there is often a group of
students who reject the FC, or rate it very poorly. They prefer normal lectures with face-to-
face contact to the lecturer (Bishop & Verleger, 2013; Yong, 2015). However, differences in
the perception of different groups of students are also conceivable. For example, the question
arises as to what extent pre-service teachers perceive the approach compared to students that do
not want to become teachers, especially in the natural sciences, where the traditional approach
of practice groups differs significantly from the idea of presence phases in the FC. Here, the
methodical and didactic design of the presence phase may offer incentives for teaching that can
later be designed by the students themselves, and could be of particular interest to pre-service
teachers.
In the school context, there is a one problem group in particular: weaker learners seem to
have problems with self-organizing their learning. Last but not least, it should be noted that the

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
326 Flipped Classroom method places great demands on independent learning - which means that
there is a risk that weaker students will be disadvantaged (see Werner et al., 2018).
Discussion
The number of teachers and lecturers implementing the FC is continuously growing.

The FC is also gaining media attention, though solid empirical evidence remains a rarity.
The Covid-19 pandemic in particular has further boosted the already high level of media
attention (see Figure 1). The limitations of normal school and university operations during the
pandemic make it necessary to implement digital alternatives, including the FC. “This crisis
also represents an opportunity for the universities, because they are sounding out what can be
done digitally and are already successfully implementing many things" (Alt, President of the
German Rectors' Conference). However, schools are struggling with the fast implementation
of digital teaching alternatives. The newspaper Der Spiegel reports that “Students consider a
lack of digitization as a 'most urgent problem’”. At the same time, traditional lectures are the
focus of media criticism. If one believes international journals, traditional lectures could be
replaced by online alternatives, even after the pandemic. For example, the New York Times
states: “Don’t Kid Yourself: Online Lectures Are Here to Stay”, while the Guardian speaks of
“‘The Netflixisation of academia’” and asks “is this the end for university lectures?”
While some progress has been made in the field of university teaching, evidence
supporting the effectiveness of the FC in school settings is still lacking, as shown in Table
3. However, it is these studies that allow teaching according to the idea of evidence-based
teaching, an educational practice based on the results of well-designed scientific studies. The
implementation and research of the FC approach is, however, often based on the initiative of
individual teachers and lecturers, leading to a lack of distinction between traditional lectures,
exercises and the innovations that come with the FC. Unclear structures, the hybridization of
FC approaches and traditional teaching make it difficult to classify previous evaluation results
and field reports. While the efforts of individual teachers are to be emphasized positively, the
evaluation of these efforts usually falls short of scientific standards (see Schmid, 2017, p. 6).
A lack of evidence and vague definitions are not a shortcoming of the FC concept, but
rather a characteristic of research in a new field. The development of a new field is mainly
characterized by the gathering of different experiences and research. Further research is needed
to further sharpen definitions and support initial results with additional evidence. Adding to
this, studies are still difficult to interpret, as there is often no clear indication of methods,
teaching content or timing. It is hardly possible to classify the extent to which different settings
implement active learning in their respective FC. In future studies, it will be necessary to clarify
what was treated and discussed in the FC, how it was done and what methods were used.
In addition to the description of the content components, it is important that future studies
are more transparent, so that measuring instruments and evaluation procedures become more
comprehensible. “This will allow us to identify which aspects, technologies and concepts of the
flipped classroom work better than others, and to form best practices, providing a springboard
for other scholars” (Sampson et al., 2018, p. 41).
Overall, only a precise and didactic description of the design, implementation and
evaluation of the FC allows statements about the effectiveness of the concept and its components
to be contrasted with traditional teaching.
“Commonly, the flipped classroom approach is taken for granted as effective in improving student
learning, and the experimental setting or flipped classroom approach used is not fully described. To
conclude, rigorous and empirically well-grounded studies currently seem to be rare in the research
on flipped classrooms. Very few studies can make generalizable or transferrable knowledge claims

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
and thereby contribute to the development of the field of interest around flipped classrooms.” 327
(Lundin et al., 2018, p.14)
Studies that assess the affective characteristics of learners by using validated items and
comparing them with traditional teaching continue to form a research gap. An investigation of
learning gains by means of control group studies has become the focus of research in recent
years, producing positive initial results. Here, too, replication and further research in various
disciplines would be desirable (especially in school settings) to strengthen trust in the FC
approach.
The present research has shown that the unclear structures and attempts at implementing
the FC are reflected in vague definitions of it (see Table 2). It has been seen that, over the
developmental course of the FC, these have changed considerably. While early definitions
highlighted e-learning and video-based learning in the self-learning phase as being core
elements, later definitions opened this up further. In order to create a comparable basis between
the implementation approaches and results of studies, the question of a satisfactory definition
must be fully clarified. An overly broad definition makes it difficult to compare studies and
implementation attempts. On the other hand, a narrow definition excludes promising approaches
that follow the core concepts of the FC. Therefore, it does not necessarily make sense to define
the FC via e-learning. Even if video learning can be integrated into an FC approach well, it
should not become its defining element. A move towards an open definition and a basic form
(see Figure 2) seems justified. The proposed definition of the FC does not limit the transfer
of knowledge to a specific medium. In a synchronous presence phase, content from the self-
learning phase is developed and consolidated by collaborative and student-centered methods,
under the supervision of the teacher or lecturer.
The study's findings suggest that the FC has not been sufficiently researched in schools to
make clear conclusions about its mode of action and effectiveness. Reviews and meta-analyses
have shown that only a few publications have met scientific standards, although the literature
base has grown considerably in the last decade. While there is much debate around the self-
learning phase, the research and discussion around face-to-face instruction in the FC falls
short. Statements about materials used, lesson designs and subject areas would make it easier
to compare studies and draw more concrete conclusions. It has been shown that the media
attention surrounding the concept and its increasingly frequent implementation as a teaching
method would imply a different, empirically validated state of research. A variety of studies
provide contrasting results. Replication studies and strongly controlled studies are needed to
assess the effectiveness of the concept in more detail.
Although methodologically varied teaching is highly desirable, lecturers and teachers
should not fall back on the concept just because it is currently being hyped by the media. The
FC can contribute to varied teaching and provide an excellent framework for the use of digital
media and cooperative forms of learning. However, it should be noted that FC use alone does not
easily solve student challenges and problems. They are instead presented with new challenges,
such as self-organized learning. Social injustices that make learning at home more difficult and
thus have a negative impact on students' performance should also be considered, especially in
the FC. For example, care should be taken to ensure that students are given the opportunity to
prepare for lessons in self-study phases at school. Overall, the analysis of previous research
suggests that the FC should be seen as a supplement to traditional instruction. The potential of
FC and the research gaps identified make the FC an attractive field of research which remains
in need of further development.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
328 Acknowledgements
The authors would like to thank the reviewers and the editorial board for their constructive
and quick feedback during the publication process. This research has received funding from the
Euratom research and training program 2014-2018 under grant agreement No 754 972.
References
Abeysekera, L., & Dawson, P. (2014). Motivation and cognitive load in the flipped classroom. Defini-
tion, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14.
https://doi.org/10.1080/07294360.2014.934336
Academy of Active Learning Arts and Sciences (AALAS) (2018). http://aalasinternational.org
Al-Zahrani, A. M. (2015). From passive to active. The impact of the flipped classroom through social
learning platforms on higher education students' creative thinking. British Journal of Educational
Technology, 46(6), 1133–1148. https://doi.org/10.1111/bjet.12353
Association of Flipped Learning Network (2014). What is flipped learning? https://bit.ly/35Gn3JO
Bart, M. (2013). Survey confirms the growth of the flipped classroom. Faculty Focus.
https://bit.ly/3mnYS8J
Baumgartner, P. (2011). Taxonomie von Unterrichtsmethoden. Ein Plädoyer für didaktische Vielfalt [Tax-
onomy of teaching methods. A plea for didactic diversity]. Waxmann.
Bergmann, J., & Sams, A. (2012, June 23-26). Flip your classroom: Reaching every student in every class
every day. International Society for Technology in Education.
Bishop, J., & Verleger, M. (2013). The flipped classroom: A survey of the research [Paper Presenta-
tion]. ASEE Conference Proceedings, Atlanta, USA. http://www.asee.org/public/conferences/20/
papers/6219/
Blair, E., Maharaj, C., & Primus, S. (2016). Performance and perception in the flipped classroom. Ed-
ucation and Information Technologies, 21(6), 1465–1482. https://doi.org/10.1007/s10639-015-
9393-5
Buechler, D. N., Sealy, P. J., & Goomey, J. (2014). Three pilot studies with a focus on asynchronous
distance education [Paper presentation]. Proceedings of 121st ASEE Annual Conference & Ex-
position, Indianapolis, USA.
Cavalli, M., Neubert, J. J., Mcnally, D., & Jacklitch‐Kuikan, D. (2014). Comparison of student perfor-
mance and perceptions across multiple course delivery modes. [Paper presentation]. Proceedings
of 121st ASEE Annual Conference, Indianapolis, USA.
Chetcuti, S. C., Hans, J. T., & Brent, J. P. (2014). Flipping the engineering classroom: Results and obser-
vations with non‐ engineering students [Paper presentation]. Proceedings of 121st ASEE Annual
Conference & Exposition, Indianapolis, USA.
Clark, R. M., Norman, B. A., & Besterfield‐Sacre, M. (2014). Preliminary experiences with “flipping” a
facility layout/material handling course. In: Y. Guan, & H. Liao (Eds.), Proceedings of the 2014
Industrial and Systems Engineering Research Conference. Montreal, Canada.
Deslauriers, L., Schelew, El., & Wieman, C. (2011). Improved learning in a large-enrollment physics
class. Science 332(6031), 862–864. https://doi.org /10.1126/science.1201783
Everett, J. W., Morgan, J. K., Stanzione, J. F., & Mallouk, K. E. (2014). A hybrid flipped first‐year engi-
neering course [Paper presentation]. Proceedings of 121st ASEE Annual Conference & Exposi-
tion, Indianapolis, USA.
Finkenberg, F. (2018). Flipped Classroom im Physikunterricht [Flipped classroom in physics teaching].
Logos Verlag.
Fogarty, J. T. (2017). Questioning the Flip: The hidden dilemmas of the greatest thing since sliced bread.
Journal of Contemporary Business Issues Special NAAS Issue 2017. https://bit.ly/3pvAIcT
Fredriksen, H. (2020). Exploring realistic mathematics education in a flipped classroom context at
the tertiary level. International Journal of Science and Mathematics Education, 19(2), 1–20.
https://doi.org/10.1007/s10763-020-10053-1

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Freeman, S., Eddy, Sarah L., McDonough, M., Smith, Michelle K., Okoroafor, N., Jordt, H., & Wen- 329
deroth, M. (2014). Active learning increases student performance in science, engineering, and
mathematics. Proceedings of the National Academy of Sciences of the United States of America,
111(23), 8410–8415.
Ghadiri, K., Qayoumi, Mohammad, H., Junn, E., Hsu, P., & Sujitparapitaya, S. (2014). Developing and
implementing effective instructional stratgems in STEM [Paper presentation]. Proceedings of
121st ASEE Annual Conference & Exposition, Indianapolis, USA.
Gilboy, Mary B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using
the flipped classroom. Journal of Nutrition Education and Behaviour, 47(1), 109–114.
https://doi.org/10.1016/j.jneb.2014.08.008
Großkurth, E., & Handke, J. (2016). Inverted classroom and beyond. Lehren und Lernen im 21. Jahrhun-
dert [Teaching and learning in the 21st century]. Tectum Wissenschaftsverlag.
He, W., Holton, A., Farkas, G., & Warschauer, M. (2016). The effects of flipped instruction on out-of-class
study time, exam performance, and student perceptions. Learning and Instruction, 45(1), 61-71.
https://doi.org/10.1016/j.learninstruc.2016.07.001
Heuett, W. J. (2017). Flipping the math classroom for non-math majors to enrich their learning experi-
ence. Problems, Resources, and Issues in Mathematics Undergraduate Studies, 27(10), 889–907.
https://doi.org/10.1080/10511970.2016.1256925
Holzer, B. (2020): An der Massenuniversität machen die Viren Party [Viruses are having a party at the
mass university]. FAZ.NET. https://bit.ly/35HLR4c
Huber, E., & Werner, A. (2016). A review of the literature on flipping the STEM classroom: Preliminary
findings. In: S. Barker, S. Dawson, A. Pardo, & C. Colvin (Eds.), Show me the learning. Proceed-
ings ASCILITE 2016 Adelaide (pp. 267-274). ASCILITE.
Hwang, G., Lai, C., & Wang, S. (2015): Seamless flipped learning: A mobile technology-enhanced flipped
classroom with effective learning strategies. Journal of Computers in Education, 2(4), 449–473.
https://doi.org/10.1080/10511970.2016.1256925
Jensen, J. L., Kummer, T. A., & Godoy, P. D. (2015). Improvements from a flipped classroom
may simply be the fruits of active learning. CBE Life Science Education, 14(1), Article 5.
https://doi.org/10.1187/cbe.14-08-0129
JIM‐Studie (2018). Medienpädagogischer Forschungsverbund Südwest [Media Education Research Net-
work Southwest] (Hrsg.). https://www.mpfs.de/studien/jim-studie/2018/
Jungic, V., Kaur, H., Mulholland, J., & Xin, C. (2015). On flipping the classroom in large first year
calculus courses. International Journal of Mathematical Education in Science and Technolo-
gy, 46(4), 1–8. https://doi.org/10.1080/0020739X.2014.990529
Kalavally, V., Chan, C. L., & Khoo, B. H. (2014). Technology in learning and teaching: Getting the right
blend for first year engineering. In Proceedings of 2014 International Conference on Interactive
Collaborative Learning, 565–570. IEEE Xplore Digital Library.
Karabulut‐Ilgu, A., Jaramillo Cherrez, N., & Jahren, Charles T. (2017). A systematic review of research
on the flipped learning method in engineering education. British Journal of Educational Technol-
ogy, 49(3), 398-411. https://doi.org/10.1111/bjet.12548
Kiat, P. N., & Kwot, Y. T. (2014). The flipped classroom experience. In: Proceedings of IEEE CSEE&T
(pp. 39-43). IEEE Xplore Digital Library.
Kim, M. K., Kim, S. M., Khera, O., & Getman, J. (2014). The experience of three flipped classrooms
in an urban university: An exploration of design principles. In: Internet and Higher Education
(pp. 37–50). https://doi.org/10.1016/j.iheduc.2014.04.003
Kircher, E. (2015). Physikdidaktik. Theorie und Praxis [Didactics of physics. Theory and practice].
Springer Spektrum.
Lage, Maureen J., Platt, Glenn J., & Treglia, M. (2000): Inverting the classroom: A gateway to cre-
ating an inclusive learning environment. The Journal of Economic Education, 31(1), 30–44.
https://doi.org/ https://doi.org/10.1080/00220480009596759
Lankford, L. A. (2013). Isn’t the flipped classroom just blended learning? Training Pros: Leighanne’s
Learning Notes. http://ileighanne.wordpress.com/2013/01/24/
Lee, K., & Lai, Y. (2017). Facilitating higher-order thinking with the flipped classroom model. A student
teacher’s experience in a Hong Kong secondary school. Research and Practice in Technology
Enhanced Learning, 12(8), 1-14. https://doi.org/10.1186/s41039-017-0048-6

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
330 Lundin, M., Bergviken Rensfeldt, A., Hillman, T., Lantz-Andersson, A., & Peterson, L. (2018).
Higher education dominance and siloed knowledge. A systematic review of flipped classroom
research. International Journal of Educational Technology in Higher Education, 15(1), 1-30.
https://doi.org/10.1186/s41239-018-0101-6
Luster‐Teasley, S., Hargrove‐Leak, S. C., & Waters, C. (2014). Transforming undergraduate environ-
mental engineering laboratories for sustainable engineering using the case studies in the sciences
instructional method [Paper presentation]. Proceedings of the 121st ASEE Annual Conference &
Exposition, Indianapolis, USA.
Margoniner, V. (2014). Learning gains in introductory astronomy: Online can be as good as face‐to‐
face. The Physics Teacher, 52(5), 298–301. https://doi.org/10.1119/1.4872414
Mason, S., Shuman, T., & Cook, E. (2013). Comparing the effectiveness of an inverted classroom to a
traditional classroom in an upper-division engineering course. IEEE Transactions on Education,
56(4), 430–435. https://doi.org/10.1109/TE.2013.2249066
McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, & LaToya
M. (2014). The flipped classroom: A course redesign to foster learning and engagement in a health
professions school. Academic Medicine: Journal of the American Medical Association. Medical
Colleges, 89(2), 236–243. https://doi.org/10.1097/ACM.0000000000000086
Ministerium für Schule und Berufsbildung, Schleswig-Holstein (2017). Bericht der Landesregierung:
Abschlussbericht zur Umsetzung des Projektes "Lernen mit digitalen Medien“ [Report of the state
government: Final report on the implementation of the project "Learning with digital media“].
https://bit.ly/2ZHbKNo
Mok, H. N. (2014). Teaching tip: The flipped classroom. Journal of Information Systems Education,
25(1), 7–11.
Newmann, G., Kim, J., & Lee, R. (2016). The perceived effects of flipped teaching and knowledge acqui-
sition. The Journal of Effective Teaching, 16(1), 52–71.
Nimmerfroh, M. (2016). Flipped Classroom ohne Videoeinsatz [Flipped Classroom without the use of
Videos]. https://bit.ly/3hEAbkS
Nouri, J. (2016). The flipped classroom: for active, effective and increased learning – especially for
low achievers. International Journal of Educational Technology in Higher Education, 13(1).
https://doi.org/10.1186/s41239-016-0032-z
O'Flaherty, J., Phillips, C., Karanicolas, S., Snelling, C., & Winning, T. (2015): The use of flipped class-
rooms in higher education. A scoping review. The Internet and Higher Education, 25, 85–95.
https://psycnet.apa.org/doi/10.1016/j.iheduc.2015.02.002
Papadopoulos, C., & Roman, A. S. (2010). Implementing an inverted classroom model in engineering
statics: Initial results [Paper presentation]. Proceedings of the 117th ASEE Annual Conference &
Exposition, Louisville, USA.
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Educa-
tion, 93(3), 223–231. https://doi.org/10.1002/j.2168-9830.2004.tb00809.x
Rotellar, C., & Cain, J. (2016). Research, perspectives, and recommendations on implement-
ing the flipped classroom. American Journal of Pharmaceutical Education, 80(2), 34.
https://doi.org/10.5688/ajpe80234
Sampson, D., Ifenthaler, D., Spector, J., & Isaías, P. (Ed.) (2018). Digital Technologies. Sustainable Inno-
vations for Improving Teaching and Learning. Springer International Publishing.
Schmid, U., Goertz, L., & Behrens, J. (2017). Monitor Digitale Bildung. Die Schulen im digita-
len Zeitalter [Monitor digital education. Schools in the digital age]. Bertelsmann Stiftung.
https://bit.ly/2E8JiwA
Şengel, E. (2016). To FLIP or not to FLIP: Comparative case study in higher education in Turkey. Com-
puters in Human Behavior, 64(1), 547–555. https://doi.org/10.1016/j.chb.2016.07.034
Spiegel (2020). Schüler halten fehlende Digitalisierung für "dringlichstes Problem"[Students regard lack
of digitization as "most urgent problem“] (2020). DER SPIEGEL, Hamburg. https://bit.ly/3c1H-
6Dw
Sharples, M., Adams, A., Ferguson, R., Gaved, M., McAndrew, P., Rienties, B., & Whitelock,
D. (2014). Innovating Pedagogy 2014: Open University Innovation Report 3. Milton Keynes: The
Open University.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
Simpson, W., Evans, D., Eley, R., & Stiles, M. (2003). Findings from the HEI “Flip” project: Application 331
issues. International. Journal of Continuing English Education and Lifelong Learning, 13, 471.
Stokel-Walker, C. (2019). „The Netflixisation of academia“: Is this the end for university lectures?
The Guardian. https://www.theguardian.com/education/2019/dec/04/the-netflixisation-of-aca-
demia-is-this-the-end-for-university-lectures
Stone, B. B. (2012). Flip your classroom to increase active learning and student engagement. In: Proceed-
ings from 28th annual conference on distance teaching and learning, Madison, WI, USA.
Strayer, J. F. (2012). How learning in an inverted classroom influences cooperation, innovation and task
orientation. Learning Environments Research, 15(2), 171-193. https://doi.org/10.1007/s10984-
012-9108-4
Sun, J. C., Wu, Y. (2016). Analysis of learning achievement and teacher–student interactions in flipped
and conventional classrooms. The International Review of Research in Open and Distributed
Learning, 17(1). https://doi.org/10.19173/irrodl.v17i1.2116
Talbert, R. (2018). Published research on flipped learning 2000-2017 [Blog Post]. http://rtalbert.org/how-
much-research-update/
Teo, T., Tan, K., Yan, Y., Teo, Y., & Yeo, L. (2014). How flip teaching supports undergraduate chem-
istry laboratory learning. Chemistry Education Research and Practice, 15(4), 550–567.
https://doi.org/10.1039/C4RP00003J
Thai, N., Wever, B., & Valcke, M. (2017). The impact of a flipped classroom design on learning perfor-
mance in higher education. Looking for the best “blend” of lectures and guiding questions with
feedback. Computers & Education, 107, 113–126. https://doi.org/10.1016/j.compedu.2017.01.003
Tolks, D., Schäfer, C., Raupach, T., Kruse, L., Sarikas, A., Gerhardt-Szép, S., Klauer, G., Lemos, M., Fi-
scher, M., Eichner, B., Sostmann, K., & Hege, I. (2016). Eine Einführung in die Inverted/Flipped-
Classroom-Methode in der Aus- und Weiterbildung in der Medizin und den Gesundheitsberufen
[An introduction to the Inverted/Flipped-Classroom method in education and training in medicine
and the health professions]. https://bit.ly/2Fz84GP
Velegol, S. B., Zappe, S. E., & Mahoney, E. (2015). The Evolution of a flipped classroom: Evidence‐
based recommendations. Advances in Engineering Education, 4(3), 1–37.
Wagner, M., Gegenfurtner, A., & Urhahne, D. (2020). Effectiveness of the flipped classroom on student
achievement in secondary education: A meta-analysis. Zeitschrift für Pädagogische Psychologie,
35(1), 11-31. https://doi.org/10.1024/1010-0652/a000274
Walvoord, B. E., Anderson, V. J. (1998). Effective grading. A tool for learning and assessment. Jossey-
Bass Publishers.
Wasserman, N. H., Quint, C., Norris, S. A., & Carr, T. (2015). Exploring flipped classroom instruction in
Calculus III. International Journal of Science and Math Education, 15(3), 545–568.
Weidlich, J., & Spannagel, C. (2014). Die Vorbereitungsphase im Flipped Classroom. Vorlesungsvideos
versus Aufgaben [The preparation phase in the Flipped Classroom. Lecture videos versus tasks].
Waxmann.
Werner, J., Ebel, C., & Spannagel, C. (2018). Flipped Classroom – Zeit für deinen Unterricht. Praxisbei-
spiele, Erfahrungen und Handlungsempfehlungen [Flipped Classroom - Time for your lessons.
Practical examples, experiences and recommendations for action]. Bertelsmann Stiftung.
Wilson, S. G. (2013). The Flipped Class. Teaching of Psychology, 40(3), 193-
199. https://doi.org/10.1177%2F0098628313487461
Wood, A. K., Galloway, R. K., Donnelly, R., & Hardy, J. (2016). Characterizing interactive engagement
activities in flipped introductory physics class. Physical Review Physics Education Research, 12
(1). https://doi.org/10.1103/PhysRevPhysEducRes.12.010140
Yelamarthi, K., Member, S., & Drake, E. (2015). A flipped first‐year digital circuits course for
engineering and technology students. IEEE Transactions on Education, 58(3), 179–186.
https://doi.org/10.1109/TE.2014.2356174
Yong, D., Levy, R., & Lape, N. (2015): Why no difference? A controlled flipped classroom study for an
introductory differential equations course. Problems, Resources, and Issues in Mathematics Un-
dergraduate Studies, 25(9-10), 907-921. https://doi.org/10.1080/10511970.2015.1031307
Županec, V., Radulović, B., Pribicevic, T., & Miljanović, T. (2018). Determination of educational
efficiency and students’ involvement in the flipped biology classroom in primary school. Journal
of Baltic Science Education, 17(1), 162-176. https://doi.org/https://dx.doi.org/10.33225/jbse/18.17.162

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
332 Zervakis, P., & Mooraj, M. (2014). Der Umgang mit studentischer Heterogenität in Studium und Leh-
re. Chancen, Herausforderungen, Strategien und gelungene Praxisansätze aus den Hochschulen
[Dealing with student heterogeneity in teaching and learning. Opportunities, Challenges, Strat-
egies and Successful Practical Approaches from Universities.]. Zeitschrift für Inklusion, 1(2).
https://www.inklusion-online.net/index.php/inklusion-online/article/view/222
Received: December 21, 2020 Accepted: March 20, 2021
Cite as: Weiß, L.-F., & Friege G. (2021). The flipped classroom: Media hype or
empirically based effectiveness? Problems of Education in the 21st Century, 79(2), 312-
332. https://doi.org/10.33225/pec/21.79.312
Lars-Frederik Weiß MEd, Scientific Assistant & PhD Student, Leibniz University Hannover,
(Corresponding author) Welfengarten 1A, 30167 Hannover, Germany.
E-mail: weiss@idmp.uni-hannover.de
Gunnar Friege Professor, Leibniz University Hannover, Welfengarten 1A, 30167 Hannover,
Germany.
E-mail: friege@idmp.uni-hannover.de

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
333
GUIDE FOR AUTHORS
ISSN 1822-7864 /print/ ISSN 2538-7111 /Online/
General Information
The publication language is English. All authors must take care of the language revision on their
own. The language must be clear and accurate. The work should be written in an impersonal style.
The editor reserves the right to send the manuscript to be reviewed. If English is a second language
for the author, please consider having the manuscript proof read and edited before submitting.
Format of Manuscripts
Title
The main research idea/problem should be reflected in the title. The title should show the nature of
the research/study. It is recommended to avoid the title of the question form. The recommended
length for a title is no more than 12 words (APA, 2.01, p. 23).
The title of the paper 14-point, bold with capital letters, align left; titles in the text (chapters)
12-point bold, not numbered; sub-titles (subchapters) 12-point, italic, not numbered. There must
be 1 empty line before and after a title or a subtitle. The text chapters must be separated by 1
empty row. The title should be concise and informative. Avoid abbreviations.
Author names and affiliations
Full names/surnames should be provided. Please indicate affiliations of the author(s). All the
e-mails should be indicated below.
Abstract
The first page of the manuscript must begin with the title of the paper and an abstract which should
be about 150 - 250 words. For the papers reporting original research, state in brief: the primary/
main aim (the research questions addressed or any hypothesis tested); the research design; the
methods and procedures employed; the number of participants; the main outcomes and results;
the conclusions drawn from these data and results, including their implications for further research
or application/practice. An abstract represents briefly a content of a text. Do not cite references in
the abstract. The abstract should grip the reader’s attention.
Keywords
Please provide 3 to 5 keywords in alphabetical order. Note that a keyword does not have to be
made of only one word. At least one of these should indicate the topic area and one should indicate
the methodology of the research.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
334 The structure of the manuscript (recommended)

The body of the text of the manuscript must generally have the following parts:
• abstract + keywords;
• introduction (it can be divided into some subchapters if needed);
• methodology of research (it is recommended to divide /rubricate);
• results of research;
• discussion;
• conclusions and/or implications;
• acknowledgements (if any);
• references (in APA style, 7th Ed);
• appendix (if any).
P.S. The structure can be different if the paper is only theoretical qualitative research.
The length of the manuscript
The manuscript should be not shorter than 6 pages (including references, tables and figures).
Manuscripts should be typed on A4, in Times New Roman 12-point font size, single-spaced, 2.5 cm
with all the margins, word-document (Word 6.0 or later) format.
Acronyms
Acronyms should be defined the first time they appear.
Tables and figures
Tables and figures should be valuable, relevant, and visually attractive. Tables (made in Word or
another software) and figures must be referred to in the text and numbered in the order of their
appearance. Each table and figure should have a complete, descriptive title; and each table column
an appropriate heading. The texts in tables and figures should be 11-point (in some cases 10) and
their width should be 12 cm at maximum. The figures should be in format .jpg (unless done in Word
or Excel); resolution 1200 dpi. Figures, tables (black and white colour, without ground-colour) and
captions should be inserted within the manuscript at their appropriate locations. All the graphics
(figures) must be editable. The font size should be Times New Roman for all figures and tables.
Figures should be carefully explained in the text and cited in numerical order.
It is the author’s responsibility to obtain permission to reproduce illustrations, tables, etc. from
other publications.
Discussion
Focus the discussion to the two important questions: What is already known about your topic?
What your research adds? It is recommended that the aim and major findings of the research
should be reminded. And then the similarities and differences of the findings with previous research
studies should be exemplified in detail.
Conclusions
This part is not a summary. Bring out the significance of your research. Show how you’ve brought
closure to the research problem, and point out remaining gaps in knowledge by suggesting issues
for further research. The main research outcome should be clearly seen.

PROBLEMS
OF EDUCATION
IN THE 21st CENTURY
Vol. 79, No. 2, 2021
References 335
The title „References“ must be used. APA style for writing references in the text and in the reference
list must be used. References in the text should be presented in parentheses (Knox, 1988; Martin,
1995). If necessary, the page can be indicated: (Martin, 1995, p. 48). The list of references should
be presented after the text.
The author should make sure that there is a strict one-to-one correspondence between the names
and years in the text and those on the list. All the references should be listed in alphabeti¬cal order
by author’s name.
For the sake of accuracy, references should always be as updated as possible, also in the interest of
the readers and researchers on the topic the paper is dealing with.
Notes
Note 1: Manuscripts in which references are not in the APAstyle will be returned without review.
Note 2: References to online sources should include the type of medium (such as “serial online”
or “monograph online”), the date of that specific reference (if applicable), the uniform resource
locator (URL), and the date that the source was accessed. A source accessed online should al-ways
be referenced accordingly, even if it is also published in printed form.
Note 3: All papers must meet the criteria of originality and scientific quality. Obviously, they must
also comply with style and format requirements. The paper will not be subject to further review,
if the manuscript is NOT WITHIN THE SCOPE and/or there is POOR USAGE OF LANGUAGE (all
manuscripts must be written in clear and grammatical English).
Note 4: Submitted papers will be assessed based on their novelty, technical quality, potential
impact, and clarity of writing.
Note 5: All papers are checked by CrossCheck system.
Editorial Board
Updated: July 2018
Website: http://www.scientiasocialis.lt/pec/

Problems of Education in the 21st Century,
Vol. 79, No. 2, 2021
Compiler Vincentas Lamanauskas

Designer Jurgina Jankauskienė
Paste-up artist Loreta Šimutytė-Balčiūnienė
English language proofreader Ilona Ratkevičienė
15 April 2021. Publishing in Quires 9.25. Edition 80.
Publisher Scientia Socialis Ltd. in cooperation with SMC „Scientia Educologica“

Donelaicio Street 29, Siauliai, Lithuania
E-mail: scientia@scientiasocialis.lt
Phone: +370 687 95668

Printing „Šiaulių spaustuvė“ Ltd.

9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt

Problems of Education in The 21st Century, Vol. 79, No. 2, 2021

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Problems of Education in The 21st Century, Vol. 79, No. 2, 2021

Uploaded by

Copyright:

Available Formats

Problems of Education in the 21st Century, Vol. 79, No.

DRAWINGS TO LEARN SCIENCE: SOME REFLECTIONS

EXAMINATION OF SYSTEM OF TRANSITION TO UPPER-SECONDARY SCHOOLS

ORGANIZATIONAL COMMITMENT LEVELS OF PRESCHOOL TEACHERS

THE EFFECT OF EXPERIENCE ON REDUCING THE OVERCONFIDENCE

STUDENTS’ PERCEPTIONS ON DISTANCE EDUCATION:

STUDENT PERCEPTIONS OF ECUADORIAN VIRTUAL PLATFORMS

POSTGRADUATE SUPERVISION PRACTICES IN EDUCATION RESEARCH AND

THE IMPACT OF TEACHER PROFESSIONAL DIARY ON THEIR REFLECTIVE,

REVIEW OF THE STATE OF METHODOLOGICAL TRENDS IN OPEN AND

THE FLIPPED CLASSROOM: MEDIA HYPE OR EMPIRICALLY BASED EFFECTIVENESS?

INSTRUCTIONS FOR AUTHORS ............................................................................. 333

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

https://doi.org/10.33225/pec/21.79.192 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Received: March 14, 2021 Accepted: April 02, 2021

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.192

Educational policy was led by Ministry of National Education (MoNE) in Turkey.

https://doi.org/10.33225/pec/21.79.194 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.194

https://doi.org/10.33225/pec/21.79.194 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Research Aim and Research Questions

Sample consisted of the 731 eighth-grade students in an Anatolian Province. Moreover,

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.194

198 Instrument and Procedures

Variables Item Type

Unconditional (Null) Model

https://doi.org/10.33225/pec/21.79.194 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Intercept 304.70 9.63 31.62* 12

Level-1 Model (Student Level)

Level-2 Model (Classroom Level)

Level-2 model focusing on the effects of classroom variables on LGS performance

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.194

Level-3 Model (School Level)

https://doi.org/10.33225/pec/21.79.194 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.194

Implications, Limitations, and Future Directions

https://doi.org/10.33225/pec/21.79.194 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.194

https://doi.org/10.33225/pec/21.79.194 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.194

Received: February 24, 2021 Accepted: April 03, 2021

Cite as: Ertem, H. Y. (2021). Examination of system of transition to upper-secondary schools

https://doi.org/10.33225/pec/21.79.194 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ORGANIZATIONAL COMMITMENT 207

LEVELS OF PRESCHOOL TEACHERS AND

Türkan Demir Balcı

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.207

https://doi.org/10.33225/pec/21.79.207 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.207

Research Aim and Research Questions

https://doi.org/10.33225/pec/21.79.207 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)

Instrument and Procedures

ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/21.79.207

Leadership Styles of Administrators According to Preschool Teachers' Perceptions

As can be seen in Table 1, organizational commitment levels of preschool teachers are

Compliance 1.74 0.59

Identification 3.81 0.77

Internalization 3.89 0.60

Table 2 shows the Leadership Styles Scale Sub-Dimensions. As shown in Table 2,

Transformational Leadership 4.08 0.59

Transactional Leadership 2.57 0.61

Laissez-Faire Leadership 1.93 0.68

https://doi.org/10.33225/pec/21.79.207 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online)