THE FLORIDA STATE UNIVERSITY

COLLEGE OF EDUCATION

EVALUATING THE IMPLEMENTATION OF THE ONLINE TUTORIAL FOR THE

UNIVERSITAS TERBUKA DISTANCE LEARNING BACHELOR DEGREE PROGRAM IN

INDONESIA

By

IDA ZUBAIDAH

A Dissertation submitted to the

Department of Educational Leadership and Policy Studies
in partial fulfillment of the
requirements for the degree of
Doctor of Education

Degree Awarded:
Summer Semester, 2013
 UMI Number: 3596613

All rights reserved

INFORMATION TO ALL USERS

The quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.

UMI 3596613
Published by ProQuest LLC (2013). Copyright in the Dissertation held by the Author.
Microform Edition © ProQuest LLC.
All rights reserved. This work is protected against
unauthorized copying under Title 17, United States Code

ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
Ida Zubaidah defended this dissertation on June 21, 2013.

The members of the supervisory committee were:

Peter B. Easton
Professor Directing Dissertation

Allan Jeong
University Representative

Jeffrey A. Milligan
Committee Member

Robert Schwartz
Committee Member

The Graduate School has verified and approved the above-named committee members, and
certifies that the dissertation has been approved in accordance with university requirements.

ii
 ACKNOWLEDGEMENTS

I would like to express the deepest gratitude to my advisor, Dr. Peter B. Easton for his

excellent supervision, guidance, advice, and patience from the very early stage of developing the

prospectus, to conducting the research, and finally to finishing this dissertation. He continually

provided me encouragement, constructive advice, and support in many ways to get this

dissertation done. I would never have been able to finish this dissertation without his persistence

help and guidance. I am indebted to him more than he knows.

I would like to thank my committee members, Dr. Jeffrey A. Milligan, Dr. Allan Jeong,

Dr. Shouping Hu for their guidance, advice, and inputs for the improvement of the research and

the dissertation writing. Special thank to Dr. Robert Schwartz who became my member of

committee and provided me encouragement and assistance to conclude my work. Without my

committee members guidance this dissertation would not have been possible.

I would also acknowledge Dr. Thomas Luschei, my former advisor, and his family who

provided me encouragement and assistance to overcome my difficulties in the early time of my

study and offered me warm environment during my stay in Tallahassee. Special thanks also go to

Matt Stanfill who has given me very kind assistance and support from the first time I came to

FSU and continually helps me whenever I have problem with my study and my inadequacy of

English to the end of my study.

Many thanks to Jimmy Pastrano, Coordinator of Graduate Studies, FSU Dept of

Educational Leadership and Policy Studies who always helped me and provided me an answer

for whatever I need during my study in FSU.

I am very grateful to the Rector of Universitas Terbuka, Prof. Tian Belawati Ph.D. who

has given me a generous opportunity and continual support to accomplish this doctoral degree. I

iii
also gratefully acknowledge other Universitas Terbuka authorities: Dr. Yuni Tri Hewindati, Ir.

Nadia Sri Damajanti, M.Ed; M.Si., Daryono, Ph.D., and Dr. Lina Warlina, Dr. Sri Harijati, and

Endang Endrawati, M.A.; who always provided the encouragement and all supports I needed

during my study. Also, my gratitude goes to IMHERE Project which provided scholarship for the

most financial support I needed for my study.

I would like to thank Irma Adnan, Nani Dianiyati, Dimas Agung Prasetyo, and Tri

Darmayanti who have helped me with the data I needed for my research, as well as to all the

tutors, administrators, and students who have been willingly involved in my study by giving me

important and meaningful data and information to complete my research.

My love and gratitude goes to my dearest son, Adrian, who has given me unconditional

understanding, support, and patience during my time away from him when he needed me the

most. He always eased my mind and always available for representing me in family duties during

my absence. Special love and prayer also goes to my beloved mother to whom I could not fulfill

her last wish to see me, and I wish her to very rest in peace. Her unconditional love and prayers

had given me strength and peace of mind more than she possibly knew.

I gratefully thank to my husband and my family for their support and encouragement

from home. Last but not least, special thanks to my best friend from high school, Danan

Hatmoro, and his family who always being there for me whenever I needed them in my good and

especially my bad times while away from home.

Finally, I would like to thank everyone else who was important to the success of the

dissertation.

iv
 TABLE OF CONTENTS

LIST OF TABLE ......................................................................................................................... viii

ABSTRACT ................................................................................................................................... ix

CHAPTER ONE: INTRODUCTION ..............................................................................................1

Background ..................................................................................................................................1
Problem of Statement ...................................................................................................................4
Purpose .........................................................................................................................................7
Research Questions ......................................................................................................................8

CHAPTER TWO: PROGRAM PROFILE AND LITERATURE REVIEW ................................10

Introduction ................................................................................................................................10
Overview of Universitas Terbuka .............................................................................................10
Historical Backdrop ................................................................................................................11
Current Infrastructure .............................................................................................................12
The Online Tutorial Program .....................................................................................................15
Genesis of Student Learning Support .....................................................................................15
Creation of Online Tutorial Program .....................................................................................16
Evolution and Current Form of the Tutorial Program ............................................................19
Transactional Distance in Distance Education ...........................................................................21
Reducing Transactional Distance through Interactivity .............................................................25
Indonesian Student Learning Styles ...........................................................................................27

CHAPTER THREE: EVALUATION DESIGN AND METHODOLOGY ..................................31

Introduction ................................................................................................................................31
Research Design .........................................................................................................................34
Online Tutorial Model Description ........................................................................................35
Determination of the Fidelity of Implementation ...................................................................38
Sampling.....................................................................................................................................40
Sampling Programs ................................................................................................................41
Sampling Students ..................................................................................................................42
Sampling Tutors .....................................................................................................................42
Sampling Administrators ........................................................................................................42
Data Collection and Instrumentations ........................................................................................43
Online Tutorial Model ............................................................................................................43
Fidelity of Implementation .....................................................................................................44
Effects of Online Tutorial Participation .................................................................................44
Stakeholder Interviews ...........................................................................................................45
Data Analysis .............................................................................................................................46
Analyzing Implementation Data.............................................................................................47
Analyzing Test Results ...........................................................................................................48
Stakeholder Interviews ...........................................................................................................52
Final Synthesis........................................................................................................................52

v
 Data Quality: Reliability, Validity, and “Trustworthiness .........................................................53
Quality of Quantitative Data ..................................................................................................54
Value of Qualitative Data .......................................................................................................55
Effects of Triangulation ..........................................................................................................56

CHAPTER FOUR: ONLINE TUTORIAL PROGRAM IMPLEMENTATION ..........................57

Introduction ................................................................................................................................57
Reconstituting Online Tutorial Model .......................................................................................57
Physical Resources .................................................................................................................58
Conceptual Resources ............................................................................................................63
Human Resources ...................................................................................................................65
Tutorial Preparation ................................................................................................................68
Session Conduct .....................................................................................................................70
Semester Wrap-up ..................................................................................................................73
Implementation of the Online Tutorial.......................................................................................73
Resources ................................................................................................................................80
Semester Preparation ..............................................................................................................81
Tutorial Process ......................................................................................................................87

CHAPTER FIVE: THE RESULTS OF THE ONLINE TUTORIAL PROGRAM .......................97

Introduction ................................................................................................................................97
Analysis of Final Course Examination Results ..........................................................................98
Descriptive Data .....................................................................................................................98
T-test Results ........................................................................................................................101

CHAPTER SIX: INTERVIEWS AND INTERPRETATION .....................................................108

Introduction ..............................................................................................................................108
Profile of Interviewees .............................................................................................................109
Student Participants ..............................................................................................................109
Tutor Participants .................................................................................................................109
Program Inputs .........................................................................................................................111
Quality Internet Connection .................................................................................................111
Program Process .......................................................................................................................112
Student-tutor Interaction in Discussion ................................................................................112
Issues of Feedback and Grading ...........................................................................................116
Workload and Schedule Concerns........................................................................................121
Program Results .......................................................................................................................127
The Overall Usefulness of the Online Tutorial ....................................................................127

CHAPTER SEVEN: CONCLUSIONS AND RECOMMENDASIONS ....................................130

Introduction ..............................................................................................................................130
Conclusions ..............................................................................................................................131
Recommendations ....................................................................................................................133
Redefining Style of Online Interaction.................................................................................133
Creating Manual of Implementation ....................................................................................134
Upgrading and Intensifying Tutor Training for the Online Environment ............................136

vi
 Reviewing and Lightening Student Workloads ....................................................................136
Better Specifying the Nature of Student Syllabus and Guideline ........................................137
Further Investigation ............................................................................................................138
Hopes ........................................................................................................................................135

APPENDICIES ............................................................................................................................140
A. INTERVIEW PROTOCOL WITH STUDENTS ...................................................................140
B. INTERVIEW PROTOCOL WITH TUTORS.........................................................................141
C. INTERVIEW PROTOCOL WITH ADMINISTRATORS .....................................................142
D. HUMAN SUBJECT COMMITTEE APPROVAL LETTER .................................................143
E. CONSENT SCRIPT FOR STUDENT INTERVIEW .............................................................145
F. CONSENT FOR TUTOR INTERVIEW.................................................................................147
G. CONSENT FOR ADMINISTRATOR INTERVIEW ............................................................149

REFERENCES ............................................................................................................................151
BIOGRAPHICAL SKETCH .......................................................................................................155

vii
 LIST OF TABLES

Table 2.1. Leading characteristics of current UT online tutorial program ....................................20

Table 3.1. Preliminary overview of online tutorial model components ........................................39
Table 3.2. Numbers of online tutorial courses and student enrollees in four UT faculties ...........40
Table 3.3. Evaluation design summary-matrix of data analysis ....................................................54
Table 4.1. Intervention model for program inputs .........................................................................59
Table 4.2. Intervention model for project processes ......................................................................60
Table 4.3. Schedule of the online tutorial ......................................................................................64
Table 4.4. Criteria of participation and grading .............................................................................73
Table 4.5. Characteristics of program model .................................................................................75
Table 4.6. Fidelity of program implementation .............................................................................77
Table 4.7. Prediction of Indonesia internet users 2011-2016 ........................................................84
Table 4.8. The growth number of enrollees to the online tutorial at FISIP ...................................84
Table 4.9. Data of academic staff/tutor at FISIP from 2008-2012 ................................................85
Table 4.10. Number and percentage of students active in online tutorial......................................89
Table 4.11. Number of students who do and do not participate in the online tutorial activities ...95
Table 5.1. Descriptive data on sample courses ..............................................................................99
Table 5.2. Descriptive data on ten sample courses final exam results .........................................100
Table 5.3. Results of independent samples t-test analysis ...........................................................102
Table 5.4. Summary t-test results and effect sizes across ten courses .........................................104
Table 5.5 Descriptive data of aggregate t-test .............................................................................105
Table 5.6 Results of aggregate independent samples t-test .........................................................105
Table 5.7. Cohen’s d aggregate effect size ..................................................................................106
Table 6.1. Profile of student participants .....................................................................................110
Table 6.2. Profile of tutor participants .........................................................................................111

viii
 ABSTRACT

Universitas Terbuka (UT), the Indonesia Open University and the 45th state university in

the country, is the only one that uses distance learning as its sole mode of delivery and

instruction. Although UT has operated for 28 years, unlike face-to-face classroom-based

education, distance education has not been considered as a fully legitimate delivery system in

Indonesia. In the mindset of the people of Indonesia, “education” necessarily implies a

conventional classroom environment; and, as a consequence, the relatively new format of

distance learning does not register as “real education.” Yet under the pressure of increased

demand for higher education, the participation in distance learning programs at UT has been

rapidly growing.

However, that popularity also poses a problem. Indonesian students have difficulty with

distance formats given lack of familiarity with the requirements of independent study and a low

level of current aptitude for reading on their own. UT has therefore developed tutorial programs

to assist distance learning students with overcoming the problem and reviewing their material.

Most are offered face-to-face but are only accessible to the minority of students living near

centers where such instruction can be delivered. With the spread of internet access in the

country, online tutoring programs have been established but have been little evaluated. This

study was devoted to assessing the quality of implementation and effectiveness of online course

tutoring for Bachelor’s degree distance learning students enrolled in UT.

Results indicate that despite difficulties of administration, the program is relatively well

implemented and in the majority of classes, online tutorial participants score better than

classmates who do not participate in the tutorial on final exams. Overall, therefore, the online

ix
tutorial program appears to be performing a real service but to be in need of better specification,

some modification of methods and closer quality control. A number of recommendations for

greater effectiveness and better service to UT students are offered in the last section of the text.

x
This dissertation is dedicated to my beloved mother and father who took leave of us while I was

away, and to my dearest son, Adrian.

xi
 CHAPTER ONE

INTRODUCTION

Background

Universitas Terbuka (UT), the Indonesia Open University and the 45th state university in

the country, is the only one that uses distance learning as its sole mode of delivery and

instruction. Although UT has operated for 28 years, unlike face-to-face classroom-based

education, distance education has not been considered as a fully legitimate delivery system in

Indonesia. In the mindset of the people of Indonesia, “education” necessarily implies a

conventional classroom environment; and, as a consequence, the relatively new format of

distance learning does not register as “real education.” Dhanarajan (2001) describes the

challenge of maintaining a distance education institution in a setting where the society believe

that distance learning is only an alternative to conventional face-to-face education method and its

delivery is considered as a supplementary function of educational institutions. When the

geographical dispersion, the immense diversity of ethnic groups and cultures and the low income

per capita that characterize Indonesia are added to the mix, it becomes evident why the effort to

develop a distance education institution like UT confronts major obstacles.

The most challenging aspect of distance learning for UT students themselves is the

requirement to study independently. This skill is necessary in a distance education learning

system, since the learner is autonomous and separated from the teacher by space and time and

because all communication is mediated (Moore, 1973). To succeed in distance education,

students must have a much greater degree of learning independence than those who work in

conventional face-to-face settings. In a study using Guglielmino’s Self-Directed Learning

1
Readiness Scale, Islam (2010) found that the level of readiness of UT students for this kind of

learning is at best “average,” meaning that they have moderate chances of succeeding but do not

assume full responsibility for planning and conducting their own learning or for evaluating their

personal needs, styles and strategies in learning.

The independent learning skill required for student autonomy in learning is further

conceptualized by Moore (1993) in his theory of transactional distance. Moore argues that the

separation of teacher and learner -- which is fundamentally “pedagogical” and only

circumstantially geographical -- leads to particular patterns of reciprocal behavior. This

separation creates “a psychological and communication space to be crossed, a space of potential

misunderstanding between the inputs of the instructor and those of the learner” (Moore, 1993,

p.22). The psychological and communications space constitutes what Moore calls “transactional

distance.” Transactional distance involves the interplay of three variables: structure (the rigidity

or flexibility educational teaching strategies), dialogue (the nature of successive interactions in

the process of teaching and learning), and learner autonomy (the extent of learner power to

determine the learning objectives and methods for accomplishing them).

Moore (1993) explains that the higher the degree of structure and the lower the level of

dialogue in an educational setting, the greater the transactional distance between learners and

teachers and the more autonomy students must exercise in their learning. Conversely, lower

structure and a higher dose of dialogue lead to smaller transactional distances and require less

learner autonomy. In sum, the greater the transactional distance, the more autonomy the learner

must manifest – that is, the more independence and the greater capacity to self-direct he or she

needs to develop.

2
 Although Moore’s concept of transactional distance refers to psychological distance,

geographical separation remains a significant factor, because it usually creates much greater

psychological distance than typically characterizes conventional education. The absence of face-

to-face meeting between students and teachers in distance education reduces the chance of

dialogue and increase the transactional distance between them. As Caladine (1993) argues,

although transactional distance refers to psychological distance, distance education students are

generally disadvantaged both by psychological and geographical distance.

The main learning support in UT course delivery is print media. Although UT has

designed the printed material in a particular format of instructional design to help distance

students by providing learning guidance, formative exercises and self-assessment directions, the

printed material cannot itself create great interactivity. It is in effect a high-structure and low-

dialogue delivery medium, and is thus characterized by a high level of transactional distance.

Complementing the printed material with additional support in audio and/or video formats, and

by computer-assisted interaction (CAI) resources and web-based material, is probably not

enough to greatly reduce the transactional distance in the learning process.

Saba & Sharer (1994) agrees with Moore’s concept that increasing dialogue decreases

transactional distance and increasing structure increases transactional distance. Student learning

supports that increase student-instructor interaction should therefore reduce the transactional

distance between the students and the teachers. In order to reduce students’ feelings of loneliness

and distance, UT in fact offers a variety of support services, including counseling and tutorials in

face-to-face and distance formats. The first of these was initiated in the early years of UT

establishment by correspondence (mail service); and others followed in the form of printed mass

media, radio, television, and face-to-face classroom tutorials. The printed mass media, radio, and

3
television tutorials are basically one-way communications, however, and only provide a very low

level of interactivity. While the correspondence tutorial is potentially a two-way communication,

that service only reaches a small number of students, since delivery is individual and few

students avail themselves of the resource.

The most interactive of these formulas – and the one preferred by students – is the face-

to-face classroom tutorial, since it mimics classroom-based education and so fulfills the desires

of students who are used to the conventional learning setting. It offers an opportunity for students

to interact directly with their instructors and their peers. The face-to-face tutorial, however, only

serves the small proportion of students who live near tutorial locations or can afford to travel

there.

Problem Statement

In its distance learning courses, the Universitas Terbuka is thus confronted with a major

problem: how to increase interactivity and effective support for students who are not used to

exercising full learner “autonomy” given the fact that the standard modes of support and

interaction available – correspondence, radio and television tutorials and face-to-face formats –

either reach few of the target audience or provide insufficient opportunity for interaction? The

problem is made doubly acute by the fact that most of UT students live in the remote areas of the

country. A learning interaction program is essential to help these “far-flung” students overcome

feelings of loneliness and of support deprivation.

The use of internet for distance learning support appears to offer one solution to this

problem. In 1999 UT initiated an electronic tutorial for students who had access to internet. The

new program created an alternative for those living around urban areas who were too busy to

4
attend the face-to-face tutorial, but was initially of limited value to students in more removed

locations having little access to electronic forms of communication.

In the meantime, however, internet access in Indonesia has both broadened and deepened

considerably. The Jakarta Globe announced earlier this year that “Indonesia expects nationwide

internet coverage by 2013” (http://www.thejakartaglobe.com/home/indonesia-expects-

nationwide-internet-coverage-by-2013/510557). Whether or not this goal is reached, access is

rapidly expanding. In the year 2000, the country counted only about 2 million internet users, but

that number had grown to 20 million by 2007, 42 million by 2010 – and it reached 55 million by

March 31, 2012, ranking Indonesia 8th in number of active users among all countries in the world

and making it one of the fastest growing in that regard, with the number of base users increasing

at 32% a year since the turn of the millennium.

Given Indonesia’s demography (with a population of 248,000,000 in July 2012), the most

recent figures for internet users still only amount to 22% of the country’s overall population –

but the coverage is certainly higher among high school graduates who are enrolled in (or

candidates for) distance learning. There is thus some confusion surrounding the exact nature and

pace of internet coverage in the country, but is at least clear that usage is expanding rapidly and

the potential for use of the medium in distance learning is also increasing.

In the first semester of 2012, the number of UT distance learning courses that offered

online tutorials reached 662, or two-thirds of the total of 974 courses provided by the

University’s four faculties. The number of students enrolling in these tutorials has also increased

significantly, reaching 122,189 in the current year. This statistic, however, does not represent the

actual number of individual students who participate in online tutorials, since students commonly

enroll in more than one.

5
 Online tutorials thus offer real hope of helping to overcome the “interaction” and

“transactional distance” barriers to effective distance education in Indonesia and perhaps of

contributing significantly to resolving the problem of low learner autonomy as well. However,

the actual process and results of these tutorials are much less clear. There is a lack of evidence

about the conduct of the new programs and about their effects, and the few reports available give

contradictory impressions. On the one hand, a notable study of the impact of online tutorials

suggests that student participation improved course completions rates and academic achievement

(Belawati, 2005). On the other hand, findings from several recent studies also indicate problems

and obstacles in program implementation while casting some doubt on results. For example, Rye

and Zubaidah (2008) find that uneven development of ICT infrastructure and unfamiliarity with

the use of Internet technology made it difficult for students in the new Province of Bangka-

Belitung to access the tutorials. Ayuni and Helmiatin (2007) reveal that many students who

enroll in online tutorials never in fact actively participated in these offerings (a finding confirmed

by Aji et al. in 2007).

These authors found that in 2004 the percentage of students enrolled in the online tutorial

who took part actively was only 28% in the 1st semester and 32% in the 2nd semester; in 2005 the

comparable rates were only 15% in the 1st semester and 45% in the 2nd semester; and in 2006

they were 31% in the 1st semester and 17% in the following semester. Students’ inactivity may

have been related to technical problems experienced at their location, to the non-responsiveness

of tutors and/or to difficulties experienced in accessing UT’s internet network (Belawati, 2005;

Maria et al., 2009).

In short, there is a pronounced need for better evaluation of the implementation and

results of UT’s online tutorials. The stakes and possible payoffs are high and the absence of a

6
sufficient quantity of reliable assessment studies has created a major impediment to good policy-

making.

Purpose

The purpose of the proposed dissertation is therefore to contribute to the basis for

informed policy-making about UT’s online tutorials by evaluating the fidelity and nature of

program implementation and the quality of its results. Limitations of time and funding mean that

the study envisaged will be more exploratory than exhaustive or definitive, but it should serve to

significantly increase and improve the database available on this critical dimension of distance

higher education in Indonesia.

“Fidelity of implementation” is a long-standing focus in the evaluation research literature

(recent examples include Farkas et al., 2012; Harvey et al., 2012; Rajan & Basch, 2012). In this

context, it is taken to be a positive attribute, because programs and projects are usually created to

try out possible solutions to persisting social problems -- and knowledge about whether and why

they work (or don’t) cannot be derived from the experience unless the proposed methods are in

fact faithfully used, Almost as much can be learned from a method that is tried and that fails to

produce the anticipated results as from one that is successful – but only if the procedure was in

fact executed with reasonable fidelity.

To say, though, that faithful implementation is a positive attribute in the evaluation

research literature does not mean either that a correctly implemented project will necessarily

produce better results than a poorly executed one or that nothing of value can be learned from

failures in implementation. Quite to the contrary, projects where staff members decide to modify

intervention methodology in innovative ways because they come to realize that these approaches

were poorly conceived or impractical in the given local circumstances may in fact have greater

7
impact than those that stick by the “letter of the law.” And failures in implementation may

themselves reveal poorly understood faults in the design of or administrative support for social

change efforts that are themselves very important to analyze. From them has in fact sprung a

whole literature devoted to probing and diagnosing implementation breakdowns (e.g. Guthrie,

2012; Ika, 2012).

As a consequence, fidelity of implementation is treated in this research as a positive

attribute – not because it is the most fundamental objective of the activity or the most important

goal to be maximized (those would certainly have to do with student learning and its later

beneficial application), but principally because it enables the program to learn more from its own

operations and so allows formative evaluation to best achieve its purpose of supporting program

improvement. At the same time, however, gaps in program implementation will themselves be

carefully analyzed and considered as potentially highly instructive.

For the same reasons, it is evident that the proposed dissertation will be both an

evaluation study and an implementation study. In fact, much of the evaluation research literature

has always been devoted to studies of the course of implementation and the lessons that may be

learned from its dynamics. The proposed effort situates itself right on that productive

intersection.

Research Questions

To guide achievement of the research purpose stated above, the following research

questions are posed:

1. What are the model and methods prescribed in the training currently given teachers

for implementation of the UT Distance Learning Bachelor’s degree online tutorial?

8
 2. How faithfully are these methods in fact implemented and respected in the conduct of

the online tutorials actually delivered to students?

3. To what degree does participation in the online tutorial seem to improve scores on the

final course exams?

4. How effective do the program’s principal stakeholders – students, tutors, and

administrators – think the present tutorial method and practice to be and what

recommendation do they make for its improvement?

The proposed study will be a management-oriented evaluation using Stufflebeam’s

general CIPP (or “CIPOO”) evaluation model.1 This model is based upon the assumption that the

most important purpose of evaluation is to improve the functioning of a program. Within this

broad scope, I will focus on documenting the actual “process” of the online tutorial program and

gaining a first set of clarified impressions of its results.

Despite all the obstacles encountered by UT in providing learning support, the online

tutorial is considered as one of the most promising strategy alternatives for present and future

enrollments. The rapid development of technology and the progressive broadening of student

access to Internet seem likely to make it the method of choice for mediating student-instructor

interaction in distance higher education in Indonesia. A comprehensive evaluation of the process

of online tutorials is therefore a critical need that the proposed study will help to meet.

1
CIPP stands for “Context, Input, Process and Product.” The more refined and subsequent formulation of
Stufflebeam’s model (CIPOO) distinguished in fact Context, Input, Process, Output and Outcome, where “output”
refers to the immediate products or results of a program and “outcome” to their larger term effects or impacts.

9
 CHAPTER TWO

PROGRAM PROFILE AND LITERATURE REVIEW

Introduction

Chapter Two is devoted to filling in some of the contextual background and conceptual

background for the proposed study. It begins with an overview of Indonesia’s Open University,

which is the host and sponsor for the online tutorial program being considered, and of its

experiments with electronically mediated distance learning and related student support services.

The chapter then continues and concludes with two essential elements of conceptual framework:

first, the theory of “transactional distance” in distance education and the measures and strategies

designed to reduce it; and, second, understandings of student learning style in an Indonesian

cultural context and its relation to distance learning.

Overview of Universitas Terbuka

Universitas Terbuka (UT) or the country’s “Open University” has been assigned the

mission of providing educational services to the fourth most populous country -- and the largest

archipelago -- in the world. It is particularly intended to meet the educational needs of those least

served by the traditional “brick and mortar” infrastructure. As an archipelagic country,

Indonesia’s territory consists 80% of water and 20% of land, the latter constituted by17,500

islands spread over more than 5,000 kilometers from west to east and more than 3,000 kilometers

from north to south. About 6,000 of Indonesia’s islands are inhabited and the total population

now exceeds 242 million people.

The decision of the Indonesian government to establish Universitas Terbuka (UT) in

1984 was an integral part of the national educational development strategy presented in the

10
Fourth Five-Year Development Plan (Rencana Pembangunan Lima Tahun IV or Repelita IV),

1984-1989. The main objective targeted by the development of a national distance education

institution was to increase the absorptive capacity of the higher education system in order to help

achieve the goals of the first Repelita IV (USAID, 1986). One consequence of the major

extension of primary to secondary education in the country over preceding years had been excess

demand for university-level training and degrees.

Historical Backdrop

The Netherlands colonial government that took control of the country from the Dutch

East Indies Company in 1816 inherited a fragmentary educational system and did little to

develop it. During the colonial period, few Indonesians were able to acquire formal education,

particularly at the university level. At the time of Indonesian independence in 1945, only around

2000 people out of a total population of 65 million – or less than 0.01% -- had graduated from

higher education institutions (USAID, 1986). With the efforts of nation building that followed

independence, educational development became, however, a priority. Increasing urgency to

expand tertiary education capacity resulted from the rapid development of primary and

secondary schools and the low absorption of school graduates on the labor market. In addition,

rapid growth in the numbers of primary and secondary schools created the need for more and

more teachers, and development of teacher training institutions became one of the prime stimuli

for increasing the capacity of higher education.

In the Fourth Five-Year Plan the government targeted increasing the proportion of 19-24

year olds entering higher education by 3.5% (Setijadi, 1986; USAID, 1986), which meant

1,522,300 new students or roughly 300,000 a year (USAID, 1986). There were, however, too

few state and private universities to achieve that goal; and it was impossible for the government

11
rapidly to develop new universities in order to accommodate the vast number of students

targeted. Building the necessary infrastructure and finding or training faculty to serve in these

institutions moreover promised to take years. It was therefore decided to establish an “open

university,” Universitas Terbuka (UT), as a more practical and economical means of ensuring

access to a large number of students in a relatively short period of time. It was felt that UT could

create partnerships with existing universities and government agencies to assist with staffing and

infrastructure without necessarily disrupting the execution of their own principal duties.

UT was launched by President Soeharto in September 1984 to accomplish the following

essential goals as Belawati (1995) points out:

a) to expand the capacity of higher education beyond the existing possibilities of

conventional universities;

b) to provide training facilities for primary and secondary teachers who wanted to

upgrade their qualification but could not leave their teaching duties; and

c) to provide training in specific areas needed for the economic and cultural

development of the country.

The target audiences for UT were people who wanted to continue their education but

could not access conventional universities because of work commitments, constraints of time and

place or other limitations on their ability to adjust to the strict schedules required in classroom-

based learning.

Current Infrastructure

UT Headquarters is now situated in Jakarta, and it is supported by 37 Regional Offices

located across the country that provide academic and administrative services for distance

learning students. The total number of full-time employees at UT is 1,841, composed of 794

12
academic personnel and 1,047 administrative staff. Besides the full-time staff, UT also hires part-

time personnel from various professions and areas of expertise – particularly course writers,

tutors, proctors, test item writers, and practicum facilitators. The number of outsourced part-time

staff currently reaches 11,000 persons. About 9,000 of them are tutors (Universitas Terbuka,

2009). In recruiting these outsourced staff, UT maintains partnerships and networks with external

government agencies including the existing conventional universities, other government

agencies, and schools from throughout the country. UT also has created several partnerships as

Zuhairi et al. (2007) record:

• with the post office and shipping companies for course material distribution;

• with reputable state Banks for facilitation of student fees payments;

• with the media electronic network to provide means of communication and

interaction with students;

• with university libraries as additional learning resources for student; and

• with local district educational institutions for practical teaching exercises, laboratory

sessions and examinations.

UT has four faculties or colleges: the Faculty of Social and Political Sciences (FISIP),

the Faculty of Mathematics and Natural Sciences (FMIPA), the Faculty of Economics (FEKON),

and the Faculty of Teacher Training and Educational Science (FKIP). The four faculties offer

thirty different Programs of Studies for the Bachelor’s degree, Diplomas and Certificates. UT has

also recently established a Graduate School offering Masters Programs in (a) Public

Administration, (b) Management, (c) Fishery Management and (d) Mathematics.

There are two types of students who study at UT under the four faculties. The first type is

basic-education-teacher students. These students are in-service teachers who study at the Faculty

13
of Teacher Training and Educational Science (FKIP) in order to upgrade their qualification to a

four-year university degree. The second type of student is non basic-education-teacher or

regular students who study at the other three faculties in various programs and degrees. The UT

website (www.ut.ac.id) lists the total number of active students on 30 April 2012 as 585,700, of

whom no less than 79.9% (467,969) are teacher trainees. The other 20.1% (117,721) are regular

students.

Print-based material is the basic learning media delivery device in UT because it is the

most accessible to all categories of students. The student can order the material online or by mail,

and the materials will be delivered to the student by the postal service. The printed material is

created in a particular instructional style known as the “complete self-learning” or “self-

contained learning” format, designed to help students work independently and achieve learning

goals on their own.

The progressive development of instructional education technology, however, has

allowed distance education to develop other kinds of learning aids to complete and enrich the

printed material. Much of this is in the form of audio, video, audio-graphic, video interactive,

computer-assisted interaction or web supplement programming. Each type of printed material is

now supplemented by one or more of the non-print resources or multimedia packages, chosen to

match and complement the characteristics of the existing course material.

In regular (non basic-teacher-education) programs, UT provides tutorials as an optional

learning support service. These tutorials may be delivered in classroom-based, radio, television,

mail correspondence, or online form. Student may choose the particular service or services that

best suit their learning style. Among these different varieties of learning support activities, only

the face-to-face and online tutorials contribute directly to the final grade that the student receives

14
for the course, since the tutorial grade in those cases is included in some fashion in final grade

calculations. The other learning support activities may of course contribute indirectly by

improving student learning and so helping participants to score better on the final exam, but they

are not sanctioned by a grade that then enters into the determination of marks.

The Online Tutorial Program

I turn now to the learning support system for UT Bachelor’s degree students (that is the

focus of the proposed research: namely, the online tutorial program).

Genesis of Student Learning Support

When Universitas Terbuka (UT) was launched in 1984, the Indonesian Government was

in a hurry and so only gave very limited time to the Preparatory Committee to design and

establish its functions (Setijadi, 1988; USAID, 1986). As a consequence, the Committee decided

to adopt the simplest system of distance education that could be developed with existing means.

Course materials were therefore mainly print-based and written by nationally-known professors

at existing reputable public and private universities in Indonesia. The print copy was then

enhanced with graphics and illustrations. Setijadi (1988) also notes that audio-cassettes were

extensively used at that time in order to highlight important parts of the course content and offer

some variety in the delivery mode.

During these early years of operation of UT, course delivery depended almost exclusively

on printed materials and the complementary attachments. The Open University offered no

official student learning support except encouragement for students to form study groups with

peers living in their area (Setijadi, 1988). It was hoped that study groups would reduce the

feelings of loneliness and isolation usually experienced by distance education learners. The study

groups were also expected to enhance learning because students would help each other to

15
understand the difficult parts of course content. Some students gathered in a group and studied

on their own while others formed groups and hired tutors to help them with their study.

More recently, UT has begun to offer other kinds of tutorials to help students in their

learning. These include face-to-face tutorials, tutorials by radio or television, and online tutorials.

Each has evolved over time to adjust to the situation and the nature of the association with other

parties involved in the provision of the learning service – and in order to increase the quality and

effectiveness of the support.

Creation of the Online Tutorial Program

Since its inception, UT has been struggling to provide effective support for students who

are not accustomed to distance learning and most of whom in fact live in rural and even remote

areas. They are thus frequently handicapped by a lack of adequate and regular transportation,

uneven and limited public infrastructure (like electricity and telephone network lines) and other

obstacles. These factors place limitations on the coverage of available student support services

and the effectiveness of most forms of tutorials. Correspondence tutorials delivered by mail, for

example, only reach and target individuals, rather than groups, and so provide no means of peer

interaction. Moreover, some students show little interest in this form of learning support. Face-

to-face tutorials are location-based and so only reach the small number of students who lived

near the tutorial sites. They are also rather expensive to produce. In addition, since most UT

students are working full time, they have difficulty attending face-to-face meetings within

regularly scheduled times (Setijadi, 1988). Tutorials delivered by radio and television are in

effect only one-way forms of communication and so offer limited interactivity. All these learning

supports continue to be offered by UT and students may access and participate in them on a

voluntarily basis.

16
 Ever since a national communications infrastructure was established in 1997 in

Indonesia, however, internet has started to become a popular medium of communication. In the

early years, the Internet network could only be accessed in governmental and private institutions

or companies. It was available to very few individual households, and those were restricted to

large urban areas. For students who had no access in their home or at the office, facilities were

available for public use in a variety of internet kiosks or cafes (known as “Warnets”) in most

Indonesian cities on an hourly rental basis. The rental cost was relatively affordable for UT

students.

Starting at the end of the 1990s, UT saw the Internet as a promising information and

communication medium to provide interactive learning support to distance learning students. It

was decided to begin by targeting students who lived in around urban areas and could afford the

necessary costs but simply had no time to attend tightly scheduled classroom face-to-face

tutorials. UT thus started to offer tutorials via the Internet in a mailing list format (Belawati et al.,

2002). This new method allowed students to interact both with their tutors and with their peers.

Although the interactions were asynchronous and feedback from tutors and other students was

delayed, this formula did help to overcome scheduling conflicts that had arisen in the face-to-

face tutorial setting. Students could access and participate in the tutorial at any time convenient

for them during the allotted schedule for online delivery.

The pilot version of the online tutorial was designed in the format of a mailing list where

the students registered using their private email addresses. Each course was supervised by one or

more tutors and assigned a separate group email list, which was subsequently moderated by the

tutor coordinator. Tutors were responsible for preparing topical materials for student discussion

using email as their mode of communication. Put simply, email was used to initiate, maintain,

17
and facilitate student interaction. Emailed materials were in the form of course and textbook

summaries, exercises, or questions.

During the duration of the pilot study, the number of courses and of participants was

quite small. Considering the limited number of students who had access to Internet and their

unfamiliarity with educational uses of that medium, UT only offered a few courses to limited

enrollments. UT did not have high expectations for the results of the initial online tutorial, but an

evaluation study conducted by Belawati et al. (2002) suggested that there was much potential

and recommended that UT continue to use Internet for tutorial purposes.

In 2002, the student learning support online tutorial switched to a type of open source-

based learning management system (LMS) software -- the Manhattan Virtual Classroom (MVC)

-- as part of upgrades to the online tutorial format (Belawati, 2005). This LMS was modified to

allow tutors to provide supplementary materials, to interact with the students asynchronously and

to facilitate discussions between tutors and students and among students themselves. From this

point on, tutors were trained both in technical matters (e.g., operation of the LMS) and in e-

tutoring methodologies prior to assuming their duties.

The tutorials could be found in UT’s web site (www.ut.ac.id), where students could

access the preferred course they wanted to join. Each course tutorial was facilitated by a tutor or

a team of tutors. The tutors were requested to prepare and provide at least eight examples of

initiation materials to trigger discussion. Nevertheless, the online tutorials were open throughout

the semester so that students could post questions, comments and responses to the questions

posed by tutors and fellow students at any time. In order to ensure quick turnaround, tutors were

requested to check their respective tutorial accounts every day and they were supposed to

respond to all posted questions promptly. The initiation materials were designed (by tutors) to

18
include resources on the elaboration of concepts, enrichment activities, summaries of certain

sections of the course materials and questions to provoke reflection. Students were informed that

their participation and the scores on assignments given in the online tutorials could contribute as

much as 10% to their final course grade. It was expected that students would be motivated to

participate in the online tutorials and that their participation would enhance their learning.

Evolution and Current Form of the Tutorial Program

Since the establishment of the UT online tutorial, the number of courses using that form

of learning support has increased every year. In the first semester of 2012, 662 of the 974 courses

produced by UT – or fully 68% -- offered online tutorials for students. The number of online

tutors is growing at a similar pace. In 2010, there were 575 tutors to facilitate online tutorials and

more have since been trained.

Unlike the tutors for face-to-face tutorials, all online tutors are recruited from among

academic staff at the relevant study programs of UT. UT cannot hire tutors from other

universities due to the fact that candidates are required not only to have the requisite content

expertise, but also to be technically competent in computer and internet operations. Tutors are

also asked to pledge their commitment and determination to complete the entire online tutorial

session. It would likely be difficult for UT to require full time work and commitment of outside

hired tutors who continue to exercise main responsibilities in their own universities. Moreover,

tutors are also required to have easy access to a computer and to internet, as UT can furnish

neither for them.

The Manhattan Virtual Classroom software used for the online tutorial starting in 2002

was replaced two years later by Moodle software, which has more features. The technical guide

for students to register, activate their accounts, access the syllabus and instructional material,

19
participate in discussions, and submit assignments is available on UT’s website. The basic

tutorial guidelines for using the new software are largely the same as they were with the previous

version. Table 1.1 sums up the leading characteristics of the UT online tutorial program.

Table 2.1. Leading characteristics of current UT online tutorial program

ONLINE TUTORIAL

General Criteria Student Participant Tutor Grading

Free of charge
Available to the
Academic staff
Final score of

students registered in from the relevant tutorial taken

Facilitated by a
the related course course in the from the
tutor or team of
program of study participation and
tutors
Registered student
assignment
who have activated
Have passed

One group consists
their email online tutor
Online tutorial
of 1-300 students
automatically training related to score can

Eight weeks within registered to the e-tutoring and the contribute 30%
the semester tutorial content of course
Eight weekly grade/score


Supported by online
discussions and forum
three assignments

On the basis of this background profile of the origin and nature of distance learning

programs in Indonesia and the inception of online course tutorials, I now turn to consideration of

two aspects of the conceptual framework and theoretical basis of the proposed dissertation: first,

the notion of “transactional distance” in distance learning, its effects and means for mitigating

those; and, second, the cultural characteristics of Indonesian learning styles and their interaction

with the distance learning format.

Transactional Distance in Distance Education

Distance education is “embedded” with certain characteristics that distinguish it from

conventional classroom-based education systems. These include the separation of learner and

20
teacher, the influence of a sponsoring educational organization, the use of technological media to

deliver content and to facilitate interaction between student and teacher, the provision of two-

way (or sometimes multi-way) communication, and the quasi-permanent absence of a physically

contiguous student learning group (Keegan, 1996). Among those traits, the fundamental

distinctive concept is the separation of student and teacher (Moore & Kearsley, 1996), which

should be thought of as “quasi-permanent” separation since the student and teacher may have

occasional contacts through face-to-face meetings or through media (Keegan, 1996).

According to Moore and Kearsley (1996), the separation by distance between teacher and

student simply means that they are not together at the same place and at the same time; and it

results in the necessity of using communication media to deliver course content and provide a

channel for interaction between them. The use of printed materials and media technologies as the

primary form of content delivery and communication is the other important characteristic

distinguishing distance education from other forms of education.

To analyze the student-teacher relationships that exist when learner and instructor are

separated by space and by time, Moore (1993) uses the concept of transactional distance first

suggested by John Dewey. Transactional distance refers to “…a psychological and

communications space to be crossed, a space of potential misunderstanding between the inputs of

instructor and those of the learner” (Moore, 1993, p. 22). Transactional distance manifests itself

in the “universe of teacher-student relationships” (p. 22), a concept that emphasizes

psychological rather the geographical distance. This universe is “shaped around” the constructs

of pedagogical structure, personal interaction between teachers and learners and the self-

directedness of the learner. The degree of transactional distance is determined by the quality of

21
dialogue, the flexibility of the structure, and the degree of learner autonomy. Moore (1993)

explains these elements as follows:

Structure (S): “Structure expresses rigidity or flexibility of the programme's educational

objectives, teaching strategies, and evaluation methods. It describes the extent to which
an education program can accommodate or be responsive to each learner's individual
needs” (p. 26).

Dialogue (D): “…an interaction or series of interactions having positive qualities those
other interactions might not have. A dialogue is purposeful, constructive and valued by
each party. Each party in a dialogue is a respectful and active listener; each is a
contributor, and builds on the contributions of the other party or parties” (p. 24).

Autonomy (A): … [it is] the extent to which, in the teaching/ learning relationship, it is the
learner rather than the teacher who determines the goals, the learning experiences, and
the evaluation decisions of the learning programme”(p. 31).

Moore (1993) argues that in learning systems, a high level of structure restrains dialogue.

When the dialogue among learners increases, the rigidity of delivery structure reduces.

According to Moore (2006), transactional distance is the functions of variables structure and

dialogue. Interplay of the two variables indicates four types of program: Program with dialogue

(+D) and structure (+S); program no dialogue (-D) but with no structure (-S); program with no

dialogue (-D) but with structure (+S); program with no dialogue (-D) but with structure (+S).

Program with low dialogue and high structure is considered as highest level of transactional

distance where students feel more distance in instructional delivery. In the contrary, educational

program that has dialogue combined with low degree of structure show the lowest level of

transactional distance. The degree of learner autonomy is defined by the combinations of the two

variables.

According to Starr-Glass (2012), Moore’s transactional distance theory was the first

comprehensive theory of distance learning interaction to be widely accepted among the scholars.

Moore’s transactional distance is one of theories appreciated by Garrison (2000). In his review

over distant education theories in the 21st century, Garrison considers Moore’s transactional

22
distance theory as,”… intuitively appealing and moves the field toward the realization of a

pedagogical theory” (p. 8). Another scholar, Jung (2001), also agrees that Moore’s theory

“provides a useful conceptual framework for defining and understanding distance education in

general and constitutes a source of research hypotheses more specifically.” (p. 527). Jung also

verifies that most of studies affirm the interplay of dialogue and structure of educational

exchange defines the degree of transactional distance. From many other scholars’ works Starr-

Glass (2012) concludes that for many years qualitative and quantitative research and studies have

confirmed Moore’s transactional distance theory have informed distance learning experience in

the inter relationship between structure, dialogue, and learner autonomy.

There are nonetheless criticisms to the theory as well. Gorsky and Caspi (2005) agree that

the concept of transactional distance makes insightful sense. However, they are in disagreement

with the Moore’s definition of transactional distance of “…., a space of potential

misunderstanding between the inputs of instructor and those of the learners” (Moore, 1993, p.

23), and they argue that transactional distance is percentage quantification of “student

misunderstanding” (Gorsky and Caspi, 2005; p. 8). They conclude the argument that they set out

in the following terms:

As the amount of dialogue (measured in terms of learner understanding) increases,

transactional distance (measured in terms of learner misunderstanding) decreases or as

understanding increases, misunderstanding decreases. Any attempt to support or to

validate the theory is meaningless, since a quantity and its inverse are being correlated”

(p. 9).

Benson & Samarawickrema (2009) suggest that the provision of learning support for

student in specific context is much determined by transactional distance management.

23
Sequentially, this management will affect to the design of its educational deliveries materials

such as learning material, assignments, and assessment. In managing transactional distance,

Benson & Samarawickrema (2009) recommend designing patterns of dialogue and structure

across a number of different units with a range of teaching and learning contexts. They suggest,

for example, that needs for learning support in classroom settings is typically modest, since

transactional distance in the face-to-face difference is low. On the contrary, in off-campus and

transnational teaching and learning setting characterized by a high level of transactional distance,

much more support is needed for dialogue and structure (+D+S).

In short, as Benson and Samarawickrema (2009) suggest, the type of support needed by

students depends on the teaching and learning setting and context. In forms of distance education

using primarily print-based materials, the level of structure of the course is quite high, whereas

the element of dialogue is very low. The materials are supposedly designed ways that allow and

encourage students to study independently. Complementing such materials with audio, video,

graphics, web-based materials or computer assisted instructional materials increases the

flexibility of learning while reducing the structure. Adding other learning supports, like any kind

of tutorial (face-to-face, electronic, or web-based) that allows the students to interact directly

with the instructor, will improve the dialogue component of course delivery. In distance

education, therefore, student learning supports play a very significant role in reducing

transactional distance and so helping students to succeed in their learning.

In distance education, the presence and live teaching strategies of the teacher are replaced

by learning materials. The printed materials can be supplemented by audio, video, computer-

assisted instruction (CAI), or web-based materials in order to support student’s independent

learning. In distance learning this teaching package takes the place of the lectures and student

24
lecture notes characteristic of conventional higher education (Seward, 1983). This teaching

package, however, cannot completely replace the lecture in conventional classroom teaching

since the package cannot include interaction between student and lecturer or among students

themselves. The absence of interaction in the teaching package must be compensated for by the

provision of learning supports.

Reducing Transactional Distance through Interactivity

Interaction is one of the most discussed topics in distance education and a primary focus

in the study of distance learning (Fulford & Zang, 1983; Hillman, Willis, & Gunawardena, 1994;

Moore & Kearsley, 1996; Vrasidas & McIsaac, 1999; Garrison & Cleveland-Innes, 2005; Mahle,

2007). In her literature review on interaction, Mahle (2007) concludes that interaction is a

primary component of any effective distance education program. In relation to transactional

distance, Moore (1973) distinguishes the concept of interaction from the notion of dialogue.

Although there might similarities, dialogue refers to “…interaction or series of interactions

having positive qualities that other interactions might have not. A dialogue is purposeful,

constructive and valued by each party” (p. 24). In the process of learning in distance education,

the interaction between student and instructor that has valuable qualities will increase dialogue

and reduce the transactional distance.

Moore (1996) acknowledges three distinct types of interaction that can influence the

process of teaching and learning in distance education: learner-content, learner-instructor, and

learner-learner interaction. Learner-content interaction is the process in which students examine,

consider, and process the course information presented during the educational experience.

According to Moore and Kearsley (1996), “Every learner has to construct knowledge through a

process of personally accommodating information into previously existing cognitive structures.

25
It is interacting with content that results in these changes in the learner’s understanding” (p. 128).

Learner-instructor interaction is communication between the instructor and the student in a

course. In the case of online learning, such interaction usually occurs via computer-mediated

communication and is not strictly limited to instructional communication that occurs during the

educational experience, but may include advising, offline communication, and personal dialogue.

Finally, learner-learner interaction is communication between two or among more than two

students in a course. Such interaction often occurs via asynchronous computer-mediated

communication, although it may include other forms of interpersonal and small group

communication, online and offline, that occurs during the period of a course.

This three-set interaction construct has been extended and adapted by subsequent

researchers in the area of distance and Web-based learning. Hillman, Willis and Gunawardena

(1994) added learner-interface interaction to reflect the growing role of technology in the

distance education process. They note that “…when dealing with any tool, it is necessary for the

user to interact with the device in a specific way before it will do his or her bidding” (p. 34).

They also distinguished between learner-interface interaction and the necessary mediation of an

interface which occurs in any interaction, by noting that for the technically-challenged learner,

the interface itself becomes “an independent force with which the learner must contend” (p. 35).

Evidence from the results of related studies show that constructive interaction has a

positive impact on student achievement. In a meta-analysis of interaction treatments in distance

education, Berdnard et al. (2009) synthesize from 74 empirical studies. In overall they find that

student achievement is associated with interactive treatment. The results of the meta-analysis

support the importance of the three types of interaction in DE: student-student interaction,

student-teacher interaction, and student-content interaction. The concept of three-type interaction

26
was popularized by Moore (1989) and will be used in analyzing data deriving from observation

of the UT online tutorials in this study.

Indonesian Student Learning Styles

In traditional classroom-based learning, some elements of learning process are provided

by the school (e.g., the classroom learning environment), some by the teacher or instructor as

facilitator, some by the lecture as a means of course content delivery and some by the

encouragement received from the instructor and classmates to help students learn successfully.

In distance education, although the institution provides learning support services for its students,

the students are considered to be adults who know their own learning needs. Students should

know what kind of environment best supports their learning, what learning style they prefer and

what factors most help to motivate and discipline them in their quest for knowledge.

People studying in a distance education environment while working or living in isolated

settings may have only themselves to rely upon. It is assumed that they are adults with aptitudes

for independent learning. Independent learning behaviors are often associated with concepts of

self-directed learning or learning autonomy. In very general terms, learner autonomy means that

students take responsibility for their own learning. This autonomy is manifested in Holec’s

(1981) concept of students determining their own objectives, defining the content and

progression of learning, selecting the methods and techniques to be used, monitoring the process

of acquisition and evaluates what has been acquired. A student’s level of learning autonomy may

determine their success in a distance education setting.

Most literature on distance education concludes that a distance education system can

ensure quality and promote learning achievement as well as a conventional one. However, as

Yang (2005) points out both the concept of distance education and the literature researching it

27
come from western society and are based on surveys and data conducted on western populations

and cultures. By comparing the cultures of distance education origin and Asian cultures, it is

skeptical that the distance educational research results in western culture and environment will

be suitable for Asian educational society (Yang, 2005).

Chalmers and Volet (1997) contest the common perceptions of students from Southeast

Asia as largely surface learners and passive participants who typically lack the skills of analysis

and critical thinking required for independent study. The authors have found in their study that

the stereotypical views derive from first impressions, whereas in fact the students persist in their

study in a Western educational environment. A number of other studies, however, generally

confirm the teacher-centered learning style preference of Asian students. Teachers represent

authority, power, and source of knowledge (Yang, 2005; Nguyen, 2011). Asian students are

seen as stereotypically quiet, compliant and good at following direction – pupils who moreover

expect their teachers to know everything (Wong, 2004; Yang, 2005).

In the case of Indonesian students, Dardjowidjojo (2001) – author of a study on teaching

and learning English -- argues that the concept of learner autonomy is not well adapted to

implementation in an Indonesian culture setting (his particular example being Javanese culture).

The Javanese are the biggest ethnic group in Indonesia and the one with the most cultural and

social influence. Although Dardjowidjojo admits the culture is changing, he argues that the

interrelation among members of Javanese society is very much determined by hierarchical

social structures: rank, social status and age in particular. Similar patterns in other Asian

societies mirror this acute consciousness of “who is older and who is younger, who has a higher

level of education, who has a lower level, who is in a higher institutional or economic position

and who is the lower, or who is teacher and who is student” (Schollon & Schollon, 1995, p.81).

28
This hierarchical relationship determines inequality and authority among the social structure in

the society. According to Dardjowidjojo (2001), in Javanese society, people with lower status

(students, children, or common people) are expected to obey (manut) and follow (miturut) the

elders (teacher, parent, or head of village). Values common to most Asian cultures dictate that,

in educational settings, students are expected to be obedient and follow the teacher’s direction

without question.

Indonesian society also places high value on harmony and tolerance. These cultural

values have historically helped to maintain the unity of a society that consists of around 300

different ethnic groups. In Javanese culture there is, for example, an element of cultural wisdom

that focuses on ewuh (the experience of being uncomfortable) and pekewuh (the experience of

feeling uneasy) and teaches its members to avoid confrontation in social communication

whenever they come to disagreement (Dardjowidjojo 2001). This cultural norm relates to the

principle of collectivism and harmony within society and the precept that the younger, the

commoner and the student have to respect the older person, the leader and the teacher. These

norms make people of lower status feel pekewuh whenever they disagree with the elders or

those are considered to occupy higher status. Such values also have a strong effect on the

educational environment. The behavior of a student who contradicts or argues with his or her

teachers can be seen as unethical and disrespectful. It is natural therefore for Indonesian

students to expect and follow “teacher-driven” styles in school and in their learning activities;

they receive knowledge from the teacher without question (Wong, 2004; Yang, 2005). At the

same time, this learning culture can create serious learning problems when students have to

study in a distance education system that requires high degree of student autonomy.

29
 Given these cultural realities, there has been understandable concern about how UT

manages the learning process of its students. The concern focuses particularly on the fact that

Indonesian society is characterized by heteronomy and social hierarchies that influence the

teaching and learning system in education. In the early days of UT, Dunbar (1991) was

skeptical about how Indonesian students, who were used to conventional face-to-face

instruction and teacher-centered learning styles, would succeed in their abrupt transfer to a

distance learning system that required independent learning skills.

UT leadership, which developed its distance learning methods from the United Kingdom

Open University system, fully realizes that the new education system requires a drastic

adjustment in student learning style. However, those responsible believe that Indonesian society

will gradually reach a better understanding of the new methodology and that students

progressively adjust their learning style to ensure their success in higher education. Like

Chalmers and Volet (1997), and Wong (2004), who studied the learning style of Asian

international students in Australia, also concludes that the style is actually contextual rather than

cultural. Wong (2004) finds in his study that the students hold to a teacher-centered style because

they are accustomed to it; but when they have to study in the Australian educational system,

which requires greater learner autonomy, they prove willing and able to change -- and eventually

they embrace the new style. For UT students, who may have difficulties with their first steps in

distance learning, it can be hoped that they will change their style to meet the requirement of a

new mode of delivery and a new pedagogy.

30
 CHAPTER THREE

EVALUATION DESIGN AND METHODOLOGY

Introduction

The evaluation is designed to gather a first set of critical data on the process and results

of the online tutorial program for the Universitas Terbuka (UT) Bachelor’s degree and to serve

the needs of decision-makers in UT. The essential research questions posed to guide the inquiry

are the following:

1. What are the model and methods prescribed in the training currently given teachers

for implementation of the UT Distance Learning Bachelor’s degree online tutorial?

2. How faithfully are these methods in fact implemented and respected in the conduct of

the online tutorials actually given to students?

3. To what degree does participation in the online tutorial seem to improve scores on the

final course exams?

4. How effective do principal stakeholders – students, tutors, and administrators – think

the present tutorial method and practice to be and what recommendation do they

make for its improvement?

The evaluation uses Stufflebeam’s general CIPP (Context, Input, Process, and Product)2

management-oriented approach as one analytical framework. Particular attention, however, is

given to the process of the online tutorial for several reasons that will briefly be elaborated later

in this section.

The CIPP evaluation model is considered the most influential management-oriented

design for evaluation and is widely used to help managers and administrators make good

2
…more recently modified to “CIPOO,” standing for the further distinction among Context, Input, Process, Output
and Outcome. See further explanations below.

31
decisions about a program or project (Fitzpatrick, Sanders, & Worthen, 2004). Evaluation is

defined as “the process of delineating, obtaining, reporting, and applying descriptive judgmental

information about object’s merit worth, significance, and probity in order to guide decision

making, support accountability, disseminate effective practices and increase understanding of the

involve phenomenon” (Stufflebeam & Skinkfield, 2007, p. 326).

The complete CIPP/CIPOO model is an evaluation framework conceived to assist

management in making decisions concerning four dimensions of a program: its context, input,

process, and products (where the latter includes both the internal results, or “outputs” of the

program and its longer-term external impacts, or “outcomes”). Stufflebeam & Skinkfield (2007)

explain the objectives of each stage of evaluation in the following terms:

• Context evaluation assesses the needs, problems, assets, and opportunities that a

program must address in order to help decision makers define goals and priorities.

• Input evaluation is devoted to assessing the adequacy of the action plans and

resources (human, material and financial) that are prepared for the program.

• Process evaluation focuses on the implementation of those plans, the actual delivery

of resources to the field and the conduct of activities carried out to produce desired

results.

• Product evaluation, on the other hand, concerns itself with the assessment of the

results of the intervention – both the immediate ones, the program’s “outputs” (e.g.,

learning gains) and the longer-term “outcomes” that represent the impact of those

immediate results, whether intended or unintended, on the subsequent well-being of

participants.

32
 In the limited time available for this study, the evaluation focuses on the critical process

or implementation dimension of the intervention: that is, the degree to which needed resources

are in fact delivered to the field, the way in which the online tutorial program is actually

conducted and the nature of students’ experiences. In the case of an online learning endeavor,

“delivery to the field” refers in particular to the internet programming actually made available

and the student activities effectively promoted or supported.

However, the other dimensions of the Stufflebeam model remain relevant to the study

and has also been the object of some inquiry, particularly during stakeholder interviews, because

diagnosis of the principal strengths and weaknesses of the program is always contingent on some

awareness of the entire scope of its operations and on at least some initial insight into the linkage

between project activities and desired results. The types of investigation to be carried out with

respect to other dimensions of the UT online tutorial program are therefore the following:

• Context: Examination of the responsiveness of the program to an accurate sense of

the needs and potentials of its clientele.

• Input: Exploration of organizational factors affecting the quality of the services

provided to students.

• Products: Collection of initial data concerning the contribution of the tutorial itself to

student success on final course examinations and their overall academic success.

The connection with results is probed via examination of the relationship between the grades that

participating students receive on the tutorial and their final course examination score. This is

scarcely a definitive measure of tutorial “output,” but it offers a first series of insights and is a

provocative element of information, because it can serve as a valuable prompt and discussion

topic for interviews with stakeholders.

33
 The online tutorial program is considered one of the most promising and progressive

forms of learning support for UT students. Although the service can only reach students who

have access to the internet, student interest in the service is steadily growing in line with the

expansion of the electronic communication infrastructure in the country. Despite the hope and

the confidence invested in the service, the implementation of the online tutorial is not free from

problems either on the student or the university side. Lack of computer and internet operation

skills, limited access to the Internet, and poor Internet network connection are some of the

problems faced by students, based on the results of previous studies. On the other hand, tutor

readiness, tutor lack of time and difficulties in system maintenance are recurrent problems on the

university side. The combination of these different obstacles can materially affect the

implementation of the tutorial and the quality of the student experience. Evaluation of the

different aspects of service delivery is therefore needed to help the online tutorial reach its goals.

Research Design

One initial and important characteristic of the proposed evaluation research is that it must

be conducted at a distance. The constraints of timing and funding for completion of this Ed.D.

dissertation do not allow me to return to Indonesia during the course of data collection and write-

up, though happily I am already familiar with much of the reality of distance learning and online

tutorials in Indonesia from my own previous and extended personal and professional experience.

Statistical data collection, class “observation” and stakeholder interviewing were all conducted

by telephonic and internet means. In a sense, however, this mode of research is particularly

appropriate when the subject itself is online and distance learning and in a situation where much

of the key data can be obtained in that fashion.

34
 Like many evaluations, the proposed study adopts a mixed method design, entailing both

quantitative and qualitative data collection as well as triangulation among the various sources of

information in data analysis. The study is nonetheless predominantly qualitative in nature and its

two main design elements may be summarized as follows:

• Quantitative dimensions. These essentially consist of comparison of the tutorial grades

and final course examination scores of students participating in the online tutorial as a

means of beginning to assess the contribution of tutorial performance to overall success

in the online Bachelor’s program and the efficacy of the tutorial program itself.

• Qualitative dimensions. The qualitative inquiry necessary to answer the research

questions above has itself several elements: (a) description of the online tutorial program

model, (b) determination of the degree of fidelity of the program actually implemented to

this model, and (c) interpretive interviews with key stakeholders. Each is briefly

described below and then further detailed in the remaining sections of Chapter Three.

Online Tutorial Model Description

Evaluation begins with a fairly complete description of the “model” of the program, plus

relatively clear specification of how it will achieve its objectives and of the methods, resources

and activities involved – in short, a well-defined portrait of the strategy of the intervention. The

scheme takes the form of a design document, a project proposal or a classic “log-frame matrix.”

Online tutorials are accessed on the UT website (www.ut.ac.id). When a student wants to

participate in an online tutorial, s/he has to register for the tutorial and activate their account. The

instructions about how to register, activate one’s account and participate in the tutorial activities

are available on the website itself. Participation in the tutorial is optional, since not all students

have access to Internet, so anywhere between one and three hundred or more may take part,

35
though effective participation averages around 100 students per course. If enrollment in the

tutorial exceeds 300 students, the group will be split in two with separate tutors. The tutorial

begins 2-3 weeks after the end of the course registration period and continues during the middle

eight weeks of the twelve-week semester.

Unlike UT face-to-face tutorials, which are given by Regional Offices with tutors hired

from local universities, online tutorials are conducted directly by staff of the faculties at UT

Headquarters in Jakarta. The tutors for online tutorials are usually the instructors of the related

courses in the relevant program study. Instructors should participate in and pass a training

session on Internet operations and Internet-based tutoring skills. A technical instruction manual

on how to run online tutorials is also available on the UT website.

Before the commencement of any tutorial, the tutor is expected to prepare tutorial plans

plus the necessary instructional materials. The first element is the Tutorial Activity Plan, which

lays out the schedule of topics and activities during the eight-week semester on the basis of the

instructional material provided by UT. The tutor posts a syllabus for the tutorial and an initiation

to it on the tutorial website before the beginning of the semester. That syllabus should include a

description and schedule of activities, study questions for each week’s online discussions and

three written assignments to be accomplished during the semester -- though in practice this

general rule may be modified and adjusted to fit the characteristics of the particular course.

Discussion questions, for example, may only be posted immediately prior to the week to which

they apply.

At the beginning of the process, the tutor explains the rules of the tutorial and introduces

participants to each other, recommending that the tutees read the main printed course material

right away or at least study each segment before the week when it is covered. In general, online

36
discussions are conducted every week in asynchronous manner: that is, students and tutors may

post anytime during the week, which lasts from early Monday morning until late the following

Sunday evening. Weekly assignments are generally posted late Sunday evening or early Monday

morning before the beginning of the following week. Students must respond within the week to

the questions posed by the tutor and may also “interact” with their peers, but I have found no

specifications about either the timing or quantity of postings expected and will need to

investigate this.

Tutors are asked to remind and encourage students on a regular basis to be active in their

discussion work and other assignments. The online tutorial system also provides a forum where

the students and the instructors may share information or discuss issues unrelated to the week’s

study topic. This forum is intended in part as a “socialization” resource to help students who are

unaccustomed to online communication and study to adjust to the new medium.

All assignments and subject-relevant discussion postings are graded by the tutor and then

combined in some fashion into the student’s overall tutorial grade, which constitutes then 30% of

their final course grade, the other 70% being represented by their score on the final course exam.

Assignments submitted and student activity on the discussion forums both count toward the final

score for the online tutorial. If a student faithfully completes all the tasks and requirements, the

tutorial score may contribute 30% to the overall course grade. A course grade is thus a

combination of the final exam score and the tutorial score. However, if the tutorial score is lower

than the final exam score, then the tutorial score is eliminated from the calculation, since it

would do nothing to improve the course grade.

The online tutorial program is supported by the technical staff of UT. Their task is to

maintain the accessibility and stability of the network. Since UT’s Internet network depends on

37
public internet network facilities, the connection also relies on the stability of the network

supplied by the external provider. The technical staff is responsible for maintaining the

steadiness of the connectivity and for tending to other technical issues that affect the conduct of

the online tutorial system. Students are also given the email address of technical staff so that

they may more easily get help with any software, internet or technical problems that they

encounter.

Determination of the Fidelity of Implementation

These partial specifications concerning the UT online tutorial model and particularly

objectives for field-level service delivery are used in Table 3.1 hereafter as a basis for

elaborating an illustrative version of the model and the potential indicators of its fidelity of

implementation. It should also serve to make clear the additional or more finely-grained

dimensions of program strategy that still need to be confirmed, detailed or filled in.

The next step in any evaluation is to compare what was planned and proposed with what

has actually transpired. This will be accomplished through my careful (and “virtual”) observation

of the process of online tutorial conduct, further described in the following sections.

Stakeholder Interviews. A variety of people are directly concerned with the conduct and the

success of online tutorials, including students themselves, the tutors charged with delivery of

related services and the administrative staff responsible for program design, resource support,

oversight and monitoring. Some of each of these categories of people was interviewed in order to

gain their perspectives on the quality of implementation of the online tutorial and the adequacy

of its results.

38
Table 3.1: Preliminary overview of online tutorial model components
Timing

Prescribed Activity Potential Indicator

Administration makes technical instruction manual for tutors available Date of availability of manual
before commencement of tutorial
Administration ensures that tutors from appropriate faculties are selected Identity and qualifications of tutors
Prior to commencement of tutorial

selected.
Administration holds training session for tutors. Conduct and conformity of trng.
Tutor attends full training session. Attendance throughout session.
Tutors passes training session Record of session results.
Tutor prepares tutorial plan for semester
Tutor designs course assignments to be carried out by students
Tutor prepares syllabus and weekly instructional material for semester

[Similar indicators to be elaborated on the basis of the prescribed activities listed]

Students enroll in tutorial at appropriate time
If tutorial enrollment exceeds 300, administration splits tutorial in two.
Tutor makes syllabus and introductory explanations available to students
Tutorial begins at start of third week of semester
Tutor posts material introducing him/herself and course
Tutor engages students in introducing themselves.
Tutor is an active presence on tutorial site for all eight weeks of semester
Tutor posts study guide/questions for each week of course before it
commences with reminders of assignment due dates
Students post responses to questions in timely manner each week.
During conduct of tutorial

Students interact on discussion boards with their peers each week.

Tutor regularly reminds students to read material, post responses to
questions, interact with peers and submit assignments.
Tutor creates and facilitates “socialization” forum.
Students submit assignments in timely manner.
Tutor furnishes feedback and grade to students on weekly discussion
performance.
Tutor furnishes feedback and grade on assignments to students in timely
manner.
Administration monitors conduct of tutorial.
Tutor encourages students to take and submit independent assessment
Administration (Technical Services) provides needed assistance to resolve
any problems of internet operation or connectivity that arise.

39
 I turn now to details of the actual execution of the proposed study, discussed in five

categories: (a) sampling; (b) variables and themes; (c) data collection and instrumentation; (d)

data analysis; and (e) data quality. A final section of the chapter is reserved for presentation of

the proposed timeline of the research

Sampling

Sampling in the proposed study concerns both the courses and students on which

performance data were collected and the individual stakeholders selected for in-depth

interviewing. It should be noted at the outset that data on enrollments and students at UT are not

always complete or easy to access, and sampling must therefore be carried out on the basis of the

best available information, and then corrected afterwards where necessary.

In the first semester of 2012, online tutorials were offered for over 600 courses among

those taught in the four faculties of UT. Table 3.2 below presents the relevant data:

Table 3.2. Numbers of online tutorial courses and student enrollees in four UT faculties

Faculties* Number of Online Total Number of

Tutorial Courses Student Enrollments
FISIP 213 62,649
FMIPA 172 4,285
FEKON 95 41,088
FKIP 182 14,208
Total 662 122,050
LEGEND:
FISIP : Faculty of Political and Social Sciences
FMIPA : Faculty of Natural Science and Mathematics
FEKON : Faculty of Economics
FKIP : Faculty of Teacher Training and Educational Studies

One should be aware that the total in the last column on the left does not represent the number of

individual students taking courses in each faculty, but rather the number of enrollments. Since a

40
single student in a given semester may enroll in up to five different courses, the unduplicated

number of individual students is thus obviously smaller.

Sampling programs

Attention of this study focuses on the Faculty of Social and Political Sciences. That unit

is chosen as the target of “elite sampling” because it has the largest enrollments of the four

faculties and so offers the greatest likelihood of including appreciable numbers of online tutorial

participants, given the relatively small proportion that those represent within the overall student

body. Ten courses among the 213 offered were selected, using size of enrollment as a prime

criterion, since it ensures more potential research subjects, but some degree of comparison

between large and smaller classes in order to bridge this source of variability, which may have a

significant impact on density of tutor-student interaction. Ten courses were selected as sample

courses. Each of these courses has between 100 and 3500 students. However, only a fraction of

them sign up for the online tutorial and an even smaller number actually participate. Though data

on these rates are another example of information that is not publicly available and can only be

pinned down during the execution of the study, between 5% and 15% of students enrolled in a

given course actually participate in the online tutorial. As a consequence, the number of students

in a tutorial varies between 10 and 450. To the degree that these estimates are accurate, the total

number of tutorial “enrollees” across the ten courses selected itself amount to more 1900 people,

though once again there are some duplication due to a single student taking part in more than one

tutorial.

These courses are given during each semester of the academic year, but for the purposes

of the proposed research it is their conduct during the last semester of 2012 that will the focus of

observation and data collection.

41
Sampling students

Student scores on the tutorial and on the final course examination will be determined and

compared in the effort to address Research Question 3 above. I analyze the scores of all students

taking online tutorials across the ten courses and examine their correlation with the students’

final examination scores; but then I select a much smaller subset of them for interviewing by

purposive random sampling.

Interviewees were chosen at random from the roster of online tutorial participants in each

of ten courses. The ten interviewees from each course were purposively chosen from different

regions with considerations of different quality of Internet connection.

Sampling tutors

The “population” of tutors for the study is composed of the ten people responsible for the

selected online tutorials. Though it would be possible to interview – at least briefly – all ten, it is

important in this phase of the study to go into some detail about both the background and the

conduct of the work, and I plan therefore to opt for data collection with a smaller group: six of

those involved. Although the tutor interviewees were intended to be selected randomly, the

selection depended on the conformity of the tutors to involve in the interview.

Sampling administrators

The relevant population of administrators – that are, the people in UT administration

most responsible for design and supervision of the courses that I observed. On the basis of this

better-detailed roster, I selected three administrators for interview, seeking people who have had

close and lasting involvement in tutorial oversight and have exercised somewhat different

responsibilities with respect to it: design, technical assistance, and field monitoring and/or

materials production. Since this phase of the inquiry principally carried out after the observation

42
of tutorial process is complete, I moreover seek people who, by their functions, seem the best

placed to comment on and clarify any particular problems in online tutorial implementation

revealed by the previous data collection. In the real data collection I selected three sampling

administrators who involved in: design, technical assistance, and faculty authority.

Data Collection and Instrumentation

Data collection was carried out in several phases corresponding to the topics and themes

indicated by the four research questions of the study recalled at the beginning of this chapter:

establishment of the UT online tutorial model, verification of fidelity and quality of

implementation, investigation of the effects of tutorial participation on final course exam success

and stakeholder viewpoints concerning tutorial quality and effects. Methods proposed for

instrument development and data collection will now be discussed in those four categories.

Online tutorial model

Clarification of the model of the program is essential to the evaluation, even if it turns out

that certain aspects of the procedure are not well specified but are, in effect, left to the discretion

of staff or tutors themselves. Table 1 above gives a first rough indication of the apparent UT

online tutorial model, but it remains vague in some respects and debatable in others. As a

consequence, a first phase of data collection focused on verification of these plans, correction of

errors in the scheme and better specification of missing parts. This work was carried out through

examination of documentation only available in Indonesia and interview of key informants likely

to be able to fill in missing detail. Though it will be essentially conducted in the early phases of

data collection, persisting gaps in the picture may only be addressed at the time of the

stakeholder interviews described below.

43
Fidelity of Implementation

Program implementation constitutes the main focus of the proposed research. Assessment

of the implementation process to answer the second research question of the study concerning

the faithfulness of the actual process of the tutorial to the method and design prescribed for the

program. The investigation was carried out by monitoring and observing the actual process of the

tutorial. This was “virtual” observation, since the tutorial itself is entirely in web format and can

only be observed by examination of the records of interaction, posting and submission in the

selected courses and during the relevant semester that are available through Universitas Terbuka.

The online transactions that take place during the tutorial sessions -- such as uploading of

instructional materials and assignments, exchanges during discussion sessions, student questions

and submissions and performance grading -- are all recorded in the electronically archived

history of the tutorial at UT, to which I had access.

Data collection in this phase of the study entails examining the record of all activities

during each of the ten courses and noting in particular (a) the degree to which those activities

match the program model; (b) the ways in which they deviate from or supplement it; (c) the

consistency of tutorial activities among the courses; (d) the intensity of interaction between tutor

and students, as well as among the students themselves and (e) the nature and quality of feedback

from the tutor. Any adjustments or modifications to the prescribed methodology that were

observed was recorded and then be a subject for further exploration in the stakeholder interviews

to be conducted in the last phase of the study.

Effects of Online Tutorial Participation

As one of the central dimensions of UT student learning services, the online tutorial

program is expected to have a positive impact on student learning, including performance in the

44
course final exam conducted at the end of the semester. Though this study cannot establish clear

causal relations between the online tutorial and other tutee characteristics (or, for that matter,

student services provided) on the one hand and indicators of academic performance on the other,

it is nonetheless important to begin investigating these relationships and to include initial

indicators of the relationship in the overall evaluation of the tutorial and in discussions of its

effectiveness with key stakeholders.

The means for doing so will be a comparison between the final course examination scores

of students in each course who do and do not actively participate in the online tutorial. The

methods for this analysis are further discussed below. Data on final course exam scores will be

obtained from the UT database.

Stakeholder Interviews

The last question addressed by the evaluation concerns the opinions and perspectives of

key stakeholders (students, tutors, and administrators) concerning the current methods and

practices of the online tutorial program and possible means for improving them. In-depth

interviews were conducted with the sample of stakeholders indicated above using protocols to

gather information about the current process of the tutorial and its relevance to the various

stakeholders’ needs and responsibilities. The interviews were conducted in the official

Indonesian language (Bahasa Indonesia), which is also the principal language of instruction at

UT, since not all the interviewees speak English. Experience suggests that the information

collected in the interviews were more comprehensive, accurate and perceptive if the interview

sessions were conducted in Indonesian language, especially with respect to the personal opinions

and feelings of the interviewees. The interviews offered an opportunity to share an initial

compilation of the data from the first three phases of the study with the stakeholders in question

45
in order to solicit their comments and their interpretation of the results and the patterns that

appear.

Data Analysis

Both qualitative and quantitative data were gathered during the study in order to provide

a basis for answering the research questions. At the end of the process, I gathered different types

of data to analyze and synthesize. Analysis of each type required its own approaches and

techniques, and then synthesis of the entire set involves other methods.

Although the study reposes to a large extent on qualitative information, the quantitative

data was an important support. Data analysis is therefore characterized as well by mixed

methods. Quantitative data analysis includes measurements based on standardized tests, rating

scales, symptom checklists, personal inventories and the like, whereas qualitative data include

interview responses, observation and field notes, permanent records, social and ethnographic

histories, etc. (Johnson & Christensen, 2008).

In qualitative data analysis, there are typically three concurrent activities: data reduction,

data display, and establishment of conclusions (Miles & Huberman, 2007). In a mixed methods

approach, however, complete data analysis includes many more steps: data reduction, data

display, data transformation, data consolidation, data comparison, and data integration (Johnson

& Christensen, 2008). Which of these steps are actually undertaken in the analysis, however,

depends on the particular characteristics and needs of the study. Since the study relies heavily on

qualitative data, the analysis methodology likewise tends to be more qualitative in nature. The

quantitative data serves more as a support and source of triangulation to be integrated in the final

analysis.

46
 Below I first discuss the different kinds of data analysis to be undertaken in the three

main phases of the study, and then address the issue of overall synthesis and conclusions.

Analyzing Implementation Data

The first two research questions of the study both concern the implementation of the

online tutorial program. Analysis here essentially entails comparing the actual conduct of the

online tutorials for the ten classes with the prescribed model initially outlined above, which is

much more fully explicated and filled in during the first phase of data collection. It should be

remembered that the detail of actual program implementation almost always differs in some

important respects from what was planned and proposed – due either to lack of clarity or

specificity in the model itself, to problems in its execution, to new methods invented or adopted

in the field as a means of resolving unanticipated problems or to some combination of those

factors. Analysis of the data on implementation thus inevitably reveals areas where the program

as carried out fulfilled expectations and others where it didn’t – or even in fact surpassed them –

for a host of reasons. And the results of this analysis creates a series of further questions (Why

the differences? How useful or necessary do they seem to have been?) that was explored during

the stakeholder interview phase of the work.

At the same time, the data from “observation” of the online tutorial sessions will also be

qualitative in nature and excerpting and coding of particularly revealing interactions and

comments will be an important strategy and one that will yield valuable elements for the

database of study results.

47
Analyzing Test Results

“Test results” here refer principally to data on the final examination scores for each of the

courses in the sample during the semester of observation. Analyses of these data are both

descriptive and comparative.

Descriptive analysis involves establishing and examining measures of central tendency

(mean, median and mode) and of variability (standard deviation, variance) that characterize the

scores, both for each of the ten courses individually and overall, along with percentages of

students in each and for all ten courses that obtain passing scores.

It is comparative analysis of the data, however, that helps us to begin answering the third

research question of the study about the effect of online tutorial participation on final course

exam scores and academic success. This essentially means comparing the results obtained by

students who participated in the online tutorial program with those of a similar “control” group

of students who did not.

Validity challenges. Genuine problems arise, however, in defining and composing that

control group. To begin with, this sort of program evaluation is not a rigorous experimental study

of the effects of tutorial participation. Conditions do not allow that. Rather, like many

evaluations, it is designed to offer some insight into the relationship between tutorials and course

performance and to use triangulation among quantitative and qualitative data on the topic to give

some added validity, reliability and trustworthiness to results.

In particular, significant threats to validity would be involved if we were simply to

compare the final exam scores of (all or a sample of) the students who participate actively in the

online tutorial with those of (all or a sample of) the much larger body of students who never

enroll in the online tutorial at all. Tutorial “enrollees” arguably constitute something of an elite

48
group of students who have easier access to internet, possess greater facility with computers and

live in more urban areas – all factors likely to be associated with increased aptitude for higher

education study and so better results on the final exams, whether or not the students in question

participate in the tutorial.

Control group composition. Ideally, I would hope to compose a more valid control

group by selecting from the large number of “regular” course enrollees (i.e., people who do not

take the tutorial) a set of students whose characteristics match those of tutorial participants with

respect to what appear to be the most telling variables – e.g., urban residence, computer facility,

academic background and perhaps gender. That list of significant characteristics is, of course,

somewhat notional, as there are no studies of the “correlates” of UT course performance to draw

upon. Moreover, very careful selection would be required, since, as mentioned, the larger body

of non-tutorial course enrollees in the ten sample classes is likely to be weighted toward the other

end of the spectrum with respect to variables like urban residence and computer facility.

With the exception of gender, however, those data on UT students are simply not

available – and so there is no way to manually assemble a control group from the larger database.

This leaves one principal course of action: use the appreciable number of students who enroll in

the online tutorial but do not actually participate in it as the pool for constitution of the control

group. Available data suggest that as many as 70% and always more than 30% of online tutorial

enrollees never take part in the session, some because they turn out to live close enough to the

site of face-to-face tutorials to attend those. Since their initial enrollment in the online tutorial

indicates at least access to and relative familiarity with computers and internet, they themselves

likely constitute a more urban group than the overall body of UT course-takers, one whose

49
relevant characteristics at least roughly approximate those of the smaller group of students who

actually complete the online tutorial.

It is therefore proposed to compare scores between a sample of students stratified by

gender among actual online tutorial participants and a comparable sample of students stratified

by gender among those who signed up for the tutorial for one or more of the ten sample courses

but never actually took part in it. To that end, I will use an independent-measures t-test approach.

T-test comparison. The general purpose of the independent-measures t-test is to

determine whether the sample mean difference obtained is a real difference between the two

populations or simply the result of sampling error. According to Gravetter & Wallnau (2007),

there are several conditions of using independent-measures t-test:

• First, an independent-measures t-test design can be used to test for mean differences

between two distinct groups or two different treatment conditions. In this case,

participation and non-participation in the online tutorial can be considered as different

treatments.

• Secondly, the independent-measures t test allows the evaluator to evaluate the mean

difference between two populations using the data from two separate samples. In this

evaluation, the sample of students who participate in the online tutorial is completely

separate from the sample of students who do not participate.

• Thirdly, the independent-measure t-test design is used in situations where a

researcher/evaluator has no prior knowledge about the two populations being

compared. In fact, at present the population means and standard deviations of the two

populations on the test are all unknown.

50
 The objective of the analysis of test data is to investigate whether the final exam score of

students who fully participate in the online tutorial is better than the scores of non-participating

students. The test was two-tailed t-test. The first population is the students who participate in the

tutorial and the second population is the students who do not participate. The null hypothesis

states that online tutorial has no effect on student performance on the final exam and that there is

thus no significant difference between the population means.

H0: u1- u2 = 0

H1: u1-u2 # 0

u1: the mean score of final examination of group student participants in the online tutorial

u2: the mean score of final examination of group student non-participants in online
tutorial

The alternative hypothesis states that there is a significant difference between the mean scores

for the two populations.

Modest ambitions. Of course, the establishment of a significant difference between the

two groups would not demonstrate conclusively and all by itself that participation in the online

tutorial is responsible for that difference, since the evaluation study does not and cannot respect

the norms of experimental design -- and because the method used to establish the online tutorial

participant and control groups is only approximate and cannot ensure that they are entirely

comparable. Moreover, there is too little data on the other characteristics of the students to make

a linear regression approach feasible. But it should be remembered in this regard that the

objective of the quantitative portion of the study is only intended to contribute a first set of data

to investigation of the academic effects of the online tutorial and that insights gained from it will

be combined and “triangulated” with those furnished by the stakeholder interviews, to which we

now turn.

51
Stakeholder Interviews

Since the interviews will be carried out in the national Indonesian language, the analysis

will start with the transcription and translation of these recordings into English. Data reduction

will commence with the selection of data from the interview records, observation notes, and

document study notes. Miles & Huberman (2007) explain that data reduction must occur

continuously throughout the process of selecting, focusing, simplifying, and transforming data in

transcription format. Data reduction will be carried out by summarizing, coding, grouping into

themes, clustering, and categorizing elements that are particularly relevant to the evaluations

questions. Data display takes the form of tables and figures. All these displays are designed to

assemble and organize information into a compact and immediately accessible database. The

display makes it easier to engage in analysis and to draw conclusions from the information.

Final Synthesis

As a last step, the varied sets of data whose analysis is described above are aligned with

each other, jointly analyzed and used as a basis for triangulation on study conclusions. Happily

the sequence of analysis outlined in the foregoing paragraphs lends itself well to this progressive

synthesis, since earlier phases in the process inform the later ones. In particular, the stakeholder

interview stage of the research offers an excellent opportunity to put together all preceding

results and focus on their meaning and interpretation, an effort that will be extended thereafter

and on the basis of the perspectives offered by stakeholder interviewees in order to reach and

justify conclusions and to make suggestions for the improvement of the UT online tutorial

program.

Conclusions and verification are accomplished by cross checking and integrating all the

data and noting the similarities, regularities, patterns, explanations, and finally drawing

52
conclusion and verifications. All the steps in data analysis finally allow the evaluator to write

justification and recommendations for online tutorial program improvement.

The overall data analysis and conclusion formulation strategy is summarized hereafter in

Table 4.3.

Data Quality: Reliability, Validity and “Trustworthiness”

One of the principal concerns in the design and conduct of any research study has to be

measures taken to ensure data quality. The notion of “quality” is of course interpreted somewhat

differently in quantitative and qualitative research. In the former case, it concerns the reliability,

validity and representativity of the data and so of the analyses performed with this information.

In the case of qualitative data, on the other hand, concepts of quality revolve around notions of

“thickness,” trustworthiness and insightfulness, which translate the need for information that gets

under the surface and helps to reveal the true nature of the human experience involved and the

likely profile of all those factors that intervene between the more schematic indicators of cause

and effect represented by quantitative variables.

A mixed method study like this evaluation research combines some aspects of both types

of data and so, in the best case, succeeds in combining their virtues as well. In an exploratory

evaluation study of this sort, moreover, there is little pretention to constructing and carrying out

an “air-tight” design. My ambition is rather to succeed in pinpointing key issues and reaching at

least “order of magnitude” estimates of implementation levels and results while forming good

hypotheses about their likely volume and nature. On this basis, I discuss briefly the separate

issues posed by the quality of quantitative and qualitative data and the effects of their

combination in the proposed study.

53
Quality of Quantitative Data

The measures for the selection of courses, of student test scores and of interviewees seem

likely to ensure at least rough “representativity” of results with respect to the overall reality of

UT online tutorial programs and their implementation. Reliability has to be a concern in a

domain like this where available official data are so scarce, difficult to access and sometime

uncertain, and in a situation where the researcher herself cannot verify things first-hand on site.

Available means for palliating these obstacles include verification of statistical data with

interviewees and, when possible, triangulation from multiples sources.

Table 3.3. Evaluation design summary-matrix of data analysis

Data Collection Data

Evaluation Question Objective Instrument
Technique Analysis
What model and To identify the Document review: - Descriptiv
methods are prescribed design, model, design program e
for the online tutorial objectives, documents,
in the training procedures, and training materials,
currently given to the strategies of the and tutor guide
tutors? online tutorial Interviews: tutors, Interview
tutor trainers protocol Qualitative
analysis

How faithfully are To identify all aspects Quasi-observation: Observatio Descriptiv

these methods in fact in the process of the On the records n protocol e
implemented and tutorial and identify history of online
respected in the any adjustment, tutorial sample
conduct of the online modification, defect, courses comparing
tutorials actually given and problems during with the model
to students? the implementation and methods
by comparing with designed program.
the design program..
Interviews: tutors, Interview
students, protocol Qualitative
administrators analysis

54
Table 3.3. – continued

Data Collection Data

Evaluation Question Objective Instrument
Technique Analysis

To what degree does To examine whether Using t-test t-statistic

faithful online tutorial has independent- quantitat
implementation of the positive impact on the measure t-test to ive
online tutorial and student achievement examine the analysis
regular participation in in the final exam. difference means
it seem to be between student
associated with participants and
success on course final non-participants in
exam? the online tutorial.

How effective do To investigate the In-depth Interview Qualitati

principal stakeholders opinions and interviews: protocol ve
– students, tutors, and perceptions of the students, tutors, analysis
administrators – think stakeholders on the administrators
the present tutorial process of tutorial;
method and practice to identify the problems,
be and what obstacles,
recommendation do expectations, and
they make for its recommendations for
improvement? the process of the
online tutorial.

Threats to validity, on the other hand, are of particular interest with respect to inferences

about statistical relationships and causal effects and may arise, for example, from

misspecification of indicator variables and/or non-comparability of experimental and control

groups. Those issues have been addressed above, however, and the measures proposed seem

adequate at least to ensure that the relationships revealed establish the worthiness of these factors

for further attention.

Value of Qualitative Data

Despite the obstacles of distance created by the necessary off-site character of proposed

methods for conducting the study, the data to collected should be relatively “thick” and

productive of relevant insight. Distance in no way lessens the opportunity for thorough and

55
intensive tracking of actual internet interactions and activities, which are the heart of an online

tutorial. The distance nature of interviews (via telephone, email or Skype) does create some

potential constraints on transparency and trustworthiness, but these can be mitigated on the one

hand by comparison and triangulation between different sets of interview results and, on the

other, by using data collected earlier in the research process as “prompts” for interviews and

persuasive references to reality.

Effects of Triangulation

The very collation of the two major sets of data – quantitative and qualitative in nature –

and triangulation between them should in addition help to compensate for weaknesses in any

individual set by itself, which is one of the typical virtues of a mixed method approach into

which evaluation studies habitually try to tap. Overall, therefore, the potential of and provision

for data quality in the proposed study seem largely sufficient to enable the research to

accomplish its central purposes.

56
 CHAPTER FOUR

ONLINE TUTORIAL PROGRAM IMPLEMENTATION

Introduction

We turn now to the data presentation and analysis portion of the dissertation. Chapter IV

focuses on data concerning the implementation of the UT online tutorial.

Data concerning implementation must themselves be divided into two sections, the first

dealing with the UT online tutorial model and the second with the way in which the program is

actually implemented and the degree to which the activities are carried out and whether the

methods used in fact match initial plans. This dual focus is necessary because the UT online

tutorial model itself is not very completely or explicitly described and verifying its dimensions

therefore necessarily became part and parcel of the data collection strategy. In order to have a

point of reference for judging the fidelity of program implementation, I first had to constitute that

point of reference.

The first section of the chapter therefore reports on the results of that verification and

“reconstitution” effort, the second presents an actual program implementation and the third

summarizes the main highlights of the chapter. It should be noted that the model documented and

the implementation activities observed all focus exclusively on the Faculty of Social and

Political Sciences (FISIP), the unit sampled for this study.

Reconstituting the Online Tutorial Model

The online tutorial has been offered to FSIP students for about 15 years now since the

time when it was first launched. Although this learning support has been continuously improved

and modified in adjustment with current demands and conditions, the basic model has remained

57
the same. The dimensions used for analyzing that model are portrayed in Tables 4.1 and 4.2

hereafter and described in the following paragraphs. I begin with categories of program inputs

exemplified in Table 4.1-- physical resources, conceptual resources and human resources – and

continue with the four dimensions of program process detailed in Table 4.2: preparatory

activities, session conduct, tutor pedagogy and semester wrap-up.

Physical Resources

Infrastructure. Computer supply and internet network have been the main devices for

essential operations of distance education institutions like UT. The Computer Center, located in

the head office of Universitas Terbuka (UT), is the center that responsible for the establishment

and sustainability of the technical electronic system and network for UT. Currently, the

electronic system and network has been focus on providing excellent supports to student

learning. E-learning support service can be classified into three groups: general support,

academic administration support, and academic support. General support includes general

information, the online bookstore, community forum, and a schedule of radio and satellite

television broadcastings. Academic administration support provides services for online course

registration and examination, and academic information on student grades and study progress.

Academic support includes independent online exercise, video on demand, supplement webs,

digital library, academic journal publication, and online tutorial.

58
Table 4.1. Intervention model for program inputs

Expectations and specifications

Source
Resp
Dom. Sub-domain Element
Quantity/regularity Quality

Existing UT
Infrastructure UT
infrastructure
RESOURCES
PHYSICAL

UT computer and online connection available

Computer with online
Equipment UT/Tut One per tutor during office hours, tutor's own home resources
connection
must be used after hours.
UT supplies available during office hours, tutor's
Supplies Incidental supplies UT/Tut As needed
own resources used after hours.
Available as part of the course material package
Curriculum US UT Available as part of the course
and can be access
Tutor Implementation Guides
CONCEPTUAL

Tutor's Implementation Guide PD UT Available but not distributed any longer

RESOURCES

distributed during training

Technical Manual on UT
Tutor's Technical Manual PD UT Can be accessed by tutors at UT website
website
Made available on UT
Students' Technical Manual PD UT Can be accessed by students at UT website
website.
These are the existing learning materials of each
Students' Course materials PD UT Brought from their course
original course.
One tutor per class but tutor Should hold a Master's degree in their program
Ed Qualifications PD Std may manage up to four classes
discipline.
Tutors simultaneously and should
Training PD Std access each class website Completed tutor training and capable of
every day. operating online software.
Registration PD Std Registered UT Bachelors students
Must have email address and internet access.
Computer access PD Std
Students Up to 300 students Should submit address and activate.
Should work online each week and complete
Participation PD Std
tasks on own schedule within week.

59
Table 4.2. Intervention model for program processes

EXPECTATIONS AND SPECIFICATIONS

SOURC
RESP
DOM. SUB-DOMAIN QUANTITY/

E
QUALITY
REGULARITY
Prepare Tutorial Concept Provides a graphic representation of essential material to Prior to beginning of semester.
Tut PD
Map be covered in course
Prepare Tutorial Activity Details sequence of coverage to course substance across
SEMESTER PREPARATION

Tut PD Prior to beginning of semester.

Plan eight Units.
Details actual subjects and activities for each week/Unit of
Prepare Tutorial Activity
Tut PD the course. Includes assignments, worksheets and tests to Prior to beginning of semester.
Units
be used.
Instructions and materials for the assignments to be given Prior to beginning of semester.
Develop Three Assignments Tut PD
to students in Weeks 3, 5 and 7.
Should contain introduction to the week's topic, links to
Develop Weekly Initiation
Tut PD extra materials or open-source resources prepared by tutor, Prior to beginning of semester.
Material
discussion questions and instructions for posting.
Upload Weekly Initiation Prior to beginning of semester.
Tut US Upload all materials needed by students.
Material and Assignments
Introduction/Welcome Tut PD Tutor orients online students to tutorial. During first session
SESSION CONDUCT

Students post answers to week's discussion question and In every weekly session.
Online discussion posting Std PD
comments on each other's postings.
Tutor logs on every day to website, reacts when No expectations specified about
Participation, supervision
Tut US appropriate to student postings and reminds those who are
and reaction reaction or interaction.
off topic or fail to post

Monitor conduct of online Vice-Dean or Online Tutorial Coordinator monitor online At least once during semester but
UT US
tutorial conduct of tutorial to verify quality and compliance no expectations specified.

60
Table 4.2. – continued
DOM. SUB-DOMAIN
Grading assignments
Grading discussion postings Tut
Providing feedback and
interim grades to students
PEDAGOGY
Essential pedagogical
TUTOR
principles to be practiced by Tut
tutors.
Total and average session
SEMESTER WRAP-UP
discussion grades
Report tutorial grades to
students
Report tutorial grades to
Examination Center
Report tutorial to the Vice
Dean of Student Support
Affairs
EXPECTATIONS AND SPECIFICATIONS
SOURC
RESP
QUANTITY/
E
QUALITY
REGULARITY
Tutor reads and grades assignments completed by each When completed
student in weeks 3, 5 and 7, while noting results.
Tutor reads and grades discussion postings by each At the end of each weekly session.
PD
student, while noting results.
No expectations
Tut PD No expectations in this regard
US No particular principles or procedures are advised. No norms established
After each session
Tut PD Tutors record the final grade of the tutorial
At end of semester
Tut PD Tutors informs the final grade of the tutorial to students
At end of semester
Tut PD Tutor report the final grades
Tut CS Should contain how tutor conduct the tutorial in the session At end of semester
61
 The Internet network connects all the buildings in the head office with fiber optics using

copper cable and wireless technology supported by two Internet service providers (ISPs). The

connection allows all the employees who work in different departments to connect and

communicate with the synergic network in the head office of the University. The wireless

network particularly serves the tutors who work at the different buildings and faculties to access

the online tutorial system at any time. The online system also allows the students to access the

online tutorial seven days a week to do the tutorial activities from anywhere in Indonesia.

In order to maintain the communication and connection with the 37 regional offices, UT

uses a virtual private network (VPN). The virtual point-to-point connection from the Computer

Center as a host to 37 regional offices enables the Center to send and receive data across with

security of private network. Related to the online tutorial, the network also provides internet

access to the tutors who work at the regional offices to maintain their activities as a tutor.

Since 2004, UT has been using Moodle as the basic platform of the online tutorial, to

replace the old system. Several factors aided in the selection of Moodle as the platform

including:

• Moodle was open source software,

• the interface design was user friendly,

• the system design and program structure were relatively easy to use and modified,

• could be easily integrated into the existing UT network system,

• stable,

• secure, and

• Moodle accommodated the principles of pedagogy for students learning.

62
 Equipment and supplies. The most essential factor influencing the success of the online

tutorial at UT is the smooth and stable accessibility of the online tutorial server. Although UT

has used the best hardware and software they can provide, the network and connection depend on

the quality of public service supplied by Internet service providers. The stability of the server,

therefore, will also rely on the quality of network offered by the provider.

The equipment needed by the tutors to support their activities managing the tutorial is the

access to the internet. It is necessary that the tutors have access to a computer and the internet

whenever they need to. The computer is needed for preparation, developing tutorial planning,

developing initiation materials, and assignments.

Access to the internet is necessary when the tutor begins the tutorial. In the beginning of

the week they have to upload the material, create discussion questions, and also upload the

assignment. During the week of the tutorial, the tutor will have to monitor and facilitate student

discussion and other activities. The need to access to the internet, therefore, is necessary for the

tutor throughout the week. The requirement to access the internet must be combined with easy

access to a computer station during their working hours.

Conceptual Resources

Curriculum. The basic curriculum for the online tutorial is the curriculum of the “parent

course” that students are taking. Online tutorials are learning supports designed to assist students

in their learning process. Online tutorials not only help students to understand the subjects in the

printed material, but also provide additional materials for student enrichment. The topics in the

tutorial are the highlight of the topics in the printed course material. The selected topics are

written in the tutorial plan.

63
 The syllabus of the online tutorial is commonly called Tutorial Activity Design

(Rancangan Aktivitas Tutorial/RAT). This syllabus is developed by the tutor and integrated with

the course and examination plans. This syllabus is designed for an eight-week tutorial. The

components in this syllabus include the name and the code of the course, the name of the tutor,

course description, course competence, weekly competence, topic, sub-topics, weekly tasks,

time, material, and references. Weekly competence and topics show what students will achieve

after completing weekly activities.

The online tutorial is an eight-week program which starts every second or third week

within the semester. The schedule of the program is designed following the general pattern

illustrated in Table 4.3 below.

Table 4.3. Schedule of the online tutorial

Week Material Activity

1 Initiation 1 Introduction
2 Initiation 2 Discussion 1
3 Initiation 3 and Discussion 2 and information of Assignment 1
Assignment 1
4 Initiation 4 Discussion 3
5 Initiation 5 and Discussion 4 and information of Assignment 2
Assignment 2
6 Initiation 6 Discussion 5
7 Initiation 7 and Discussion 6 and information of Assignment 3
Assignment 3
8 Initiation 8 Discussion 7 and reminder on the final
examination.

Instructional material and aids: Since the initiation of the tutorial online, the tutorial

system has been adapted to several software programs. The basic model of implementation of the

program has been the Implementation Guide of 2002. This guide is only for internal office use.

64
The other source of information about the online tutorial is the guide on how to operate the

software required for the activities of the online tutorial in the university website, either for tutor

or student.

Since 2002, however, the online tutorial system has been continually developed along

with alterations of the platform software. Some modifications have been made to adjust to the

current situation in terms of the number of participants, number of courses offered the tutorial,

and number of tutors. In adjustment with the dramatic increase of ICT and extensive student

enrollment to the program, a team has been developing the modified model of the online tutorial.

However, there is no result or product that can be reviewed yet for information addition to this

research.

Human Resources

Human resources that are directly involved in the implantation process of the online

tutorial include technical administrators who work under the Computer Center, faculty

administrators and tutors who are under authority of the Faculty.

Technical administrators. Technical administrators are the parties responsible for

maintaining the technical operation of the online tutorial system. The technical administrator for

the online tutorial supervises a team under the Computer Center at the University’s main campus

including staff members that have expertise in information and communication technology

(ICT). This team’s responsibilities include:

• Designing and developing the most suitable online tutorial system in the University

by considering the suitability the system with the public ICT infrastructures, the

average facility of the student network program to access the University system, and

the University budget.

65
 • Maintaining the existing system for excellent service to the students who join the

online tutorial.

• Updating the system in line with advancement of ICT to better serve the students.

• Updating the knowledge and skills of the tutors to optimize the benefits from the

system to assist students in learning.

• Helping the students when they have technical problems in the online tutorial or other

issues related to the technical matters

Faculty administrator. The faculty administrator is the person who is responsible for

managing and maintaining the online tutorial within the University. The Vice Dean of Student

Support Affairs is the person who is charged with the daily operational of the online tutorial in

the Faculty. The Vice Dean is responsible for:

• Making decisions related to policy and regulation in the Faculty related to online tutorial;

• Selecting qualified tutors for all courses;

• Facilitating the tutors with necessary training or workshops to update the tutors skills and

knowledge.

• Together with Heads of the Program, supervising the process of the tutorial.

• Evaluating the process of the tutorial.

Tutor/Instructor. The instructor or tutor is an academic staff member who has relevant

discipline with course and works in the relevant field. The instructor may work at the head office

or at any regional office. To be eligible as an instructor, the staff member has to pass at least two

trainings, general tutor training and e-tutoring training.

General Tutor Training is the basic training for all UT’s tutors. The training aims to

produce tutors who have effective and efficient teaching skills in the face-to-face tutorial and the

66
online tutorial. After completing the training, the tutors are expected to have andragogy,

personality, and professional and social competences which will be practiced in the tutorial. The

knowledge and skills given in the training includes:

a. Overview of distance education

b. How to create a concept map of the course

c. Tutorial models

d. How to develop the tutorial planning

e. How to conduct the online tutorial

f. How assess and score the tutorial

In this training the tutors will have to develop a tutorial plan which integrates with the

course and examination plan of the course. The plans are for the entirety of tutorial which later

will be used as the syllabus in the tutorial including the Tutorial Activity Design (a.k.a.

Rancangan Aktivitas Tutorial/RAT) and the weekly tutorial activity plan called Tutorial Activity

Unit (a.k.a. Satuan Aktivitas Tutorial/SAT).

E-tutoring training is training for tutors to become familiar with the operation of Moodle.

In this training the tutors learn about the overview of the software, how to activate, to access, to

upload questions and materials in multimedia, how to lead the discussion, to respond to

questions, and to conclude the tutorial. In addition, in the training the tutors also learn how to

develop initiation material, discussion questions, and assignment material.

This basic e-tutoring training is conducted for all new tutors. In addition every semester

before the tutorial begins all the tutors are provided an in-service training to discuss any new

features of the online tutorial program.

67
 According to the Implementation Guide of Online Tutorial First Edition 2002, a tutor has

the following responsibilities:

• Developing online tutorial plans

• Developing and providing eight initiation materials

• Developing three tutorial assignments to be completed by students

• Uploading the weekly initiation question and additional material, if any

• Accessing the tutorial website everyday

• Responding to questions or inquiries from students

• Scoring the assignments

• Grading the online tutorial

• Sending the students tutorial grades to the Examination Center

• Posting the final tutorial grade to the students

This concludes the tour of planned program “inputs” and we pass on now to an examination of

the “processes” by which those resources are used. That portion of the model is recapitulated in

Table 4.2 above, and each item is described in greater detail below.

Tutorial Preparation

Tutorial preparation activities are those that must be carried on before the semester

actually begins and students are allowed to access the tutorial online. In general, it is the tutor

who makes sure that all documents and materials in the tutorial are available.

Tutorial planning: Concept map, activity design and tutorial units. The Tutorial

Activity Design (Rancangan Aktivitas Tutorial/RAT) is a plan of tutorial activities within a

semester. The plan will be used as a guide to create tutorial activities in each week of tutorials.

Tutorial activity planning consist of a short description of the course, general competence

68
achieved by students after completing the course, specific competences achieved by students in

each week session of the tutorial, topics in each specific competence, identification of sub-topics

in each topic, the model tutorial, assignments and grading system, the duration of each tutorial

session, and the necessary sources and references. This is available to let the students know what

they will do and achieve in each session of the tutorial and after they complete the tutorial.

Tutorial Activities Unit (Satuan Aktivitas Tutorial/SAT) is activity planning for each

weekly session of the online tutorial. The unit activity of tutorial will be used as a tutorial guide

in one session of tutorial. Tutorial unit activities are broken down from the tutorial activity

design which will be used in every week of the online tutorial. In each session the tutor has to

demonstrate the objective subject, relevance and the usefulness of the tutorial.

Tutorial guides. The Tutorial guidance documents that should be available are the

Tutor’s Implementation Guide, Tutor’s Technical Guide, and Student’s Technical Guide.

The only implementation guide that exists as a written document is the Implementation

Guide 2002. To some extent this document is not relevant anymore to the current situation.

However, the basic rules of the online tutorial are still adopted from the guide such as the eight

week discussion session and three assignments that have to be completed by the students. New

additional rules that adjust with the current situation are usually added in the regular meeting of

online tutorial between tutor and administrators. This guide is not officially distributed any

longer due to the outdated information in the guide. Currently, a new implementation guide is

now under development.

The content of tutor’s technical guide and student’s technical guide are principally

similar. They can be seen on the UT website. The guides consist of the information on how to

69
activate the account and to operate the program in the online tutorial. Tutors and students usually

refer to these guides when they are asked about the guidance they use in the tutorial.

Student enrollment. All the students who register in a course are eligible for its online

tutorial as long as they submit and activate their email address. The students who enroll in one

course and submit the email address will be automatically grouped and set by a computer

program. The process of students joining a tutorial group involves several steps:

a) Student registers in a course or courses.

b) Student submits an email address.

c) The student activates the account with the email

d) The student will join into a group of a maximum of 300 student members in a class of

an online tutorial. If the group members exceed 300 students, the next student will be

separated into a new group, and so forth.

e) The participation of the student in the group is voluntary. There is no obligation for

the student to participate in the discussions or submit assignments although they are

registered and placed in a class for the tutorial. If the student wants the optimal score,

however, he/she has to complete all tasks in the tutorial.

Session Conduct

The sequence and style of activities in the online tutorial are characterized below.

Introduction. The first week of the online tutorial is the introduction. There is no

obligation for the tutor and the students to formally introduce each other. The basic personal

information such as name, email, city where the student lives, and student picture (optional) is

written in the site so that every participant can see each other. The first week is also time for

orientation of the tutorial to the students. The tutor informs students about the competencies that

70
will be achieved after complete participation in the tutorial, the scenario and strategy of the

tutorial, assignments which will have to be completed by the students, and an explanation about

grading system in the tutorial.

Online postings, discussions, participation and monitoring. Discussion is conducted

every week, facilitated by a tutor or a team of tutors. However, this discussion session can be

adjusted by the tutor in relevance to the nature of the course and the number of student

participants.

In weekly discussions, the tutor provides some material from the course materials or

other support materials to be read by the students. Based on the topic and the material, the tutor

will give students a question or a case to be discussed. The tutor will instruct the students what to

do in the discussion. The tutor facilitates the discussion by giving an idea, comment or remark to

keep the discussion running. The participation of the students in the discussion will be monitored

by the tutor to measure the level of activity in the tutorial which will cumulatively be graded as

the participation score.

To facilitate discussion, a tutor has to have some teaching skills such as questioning,

reinforcing, teaching variation, explaining, maintaining the discussion, and opening and closing

the discussion.

Assignments, feedback and grading. Online tutorial assignments are an academic

activity that has to be accomplished by the students to measure the level of understanding or skill

related to the subject matter given in the tutorial.

The assignment usually is a short essay that can be in the form of reading, summarizing,

problem solving, creating a model, observing an object or event, field study, or writing an

academic paper.

71
 Assignments are given three times in the course of the eight-week semester:

• Assignment on week 3 to measure the students learning accomplishment on week 1

and week 2.

• Assignment on week 5 to measure the students learning accomplishment on week 3

and week 4.

• Assignment on week 7 to measure the students learning accomplishment on week 5

and week 6.

The tutor should provide clear information on how the students should complete the assignments.

The assignments have to be clear in terms of product, scoring, and duration.

There is no clear guidance how tutors should provide feedback for the weekly discussion

and assignments. There is no detailed information regarding how the feedback in the discussion

and assignments should be given by tutors. The roles of the tutor are detailed as follows in the

Guide:

• respond to questions from students,

• motivate students to participate actively in the weekly discussion,

• respond or comment on unclear responses in the discussion within three days,

• score the assignments,

• provide students’ their final grade for the online tutorial, and

• inform the final score of the tutorial to the students.

In general, some information is given regarding tutorial scoring and grading. However,

the tutors have authority to assess students’ levels of participation on their own. The general

guidance is illustrated in Table 4.4 hereafter:

72
Table 4.4. Criteria of participation and grading

Criteria Contribution
Passive participation: accessing the website, only passively read the 20 %
initiation materials, questions, and responses from tutor and other
students.
Active Participation: Actively participate by posting comments, 30 %
responding to questions and comments from tutor and other students.
Completing 3 assignments 50%
Total 100%

The final score of online tutorial is calculated from the combination of student participation in

the discussions and scores of assignments. The grading is according the formula:

3A+7P
TS= --------
10
TS= tutorial score, A=assignment, P=participation in the discussions

Semester wrap-up

At the end of the semester-long tutorial session, tutors have to complete scores for

student participation in discussions and performance on assignments. Student scores should be

reported to the students through the given box in its tutorial site. The tutor also has to report the

students’ score to the Examination Center to be calculated with other components of grading to

produce the student final course grade.

Lastly, the tutor has to write a report to the Vice Dean of Student Support Affairs in the

Faculty including the process of the tutorial, the obstacles faced by the tutor and students and

other necessary information the Faculty needs to know.

Implementation of the Online Tutorial

The pages above describe various dimensions of the online tutorial design. How

faithfully has it in fact been implemented in practice? The section to follow presents and reviews

73
data on fidelity of implementation – both in the form of synoptic Tables 4.5 and 4.6 immediately

hereafter and of the related text on this and the following pages.

Table 4.5. presents the nature of the online tutorial model – that is, the resources that

should be aligned and the activities that should be carried out in conducting the tutorial,

according to UT standards. Table 4.6, on the other hand, summarizes data about the actual

fidelity of implementation of the program: the degree to which the various resources and

activities were in fact brought to bear or conducted. This assessment, based on the researcher’s

observations, is characterized in two ways in the columns labeled “rating” (RTG) and “variation”

(VAR). The first indicates whether the degree of correspondence between the model and actual

practice was observed to be low, medium or high, where those terms are given the following

meaning:

Low = Less than 40% fidelity to the model

Medium = 40%-70% fidelity to the model

High= 71%-100% fidelity to the model

For example, the sub-domain of “Prepare Tutorial Activity Plan” has a “High” rating because all

of the tutors (100%) develop and upload the Tutorial Activity Plan to the program sites; while

“Online discussion posting” has “Medium” rating because only about 40-50% students actively

participate in the discussions.

At the same time, the second “variation” column indicates whether this degree of fidelity

was similar among courses or differed markedly between them – and, once again, the scale used

ranges from “Low” to “High.” In developing and administering assignments, for example, tutors

may all stick with the same content (low variation) or each may create their own style of

assignments attuned to their particular circumstances (high variation); or in the discussion

74
Table 4.5. Characteristics of program model

EXPECTATIONS AND SPECIFICATIONS

SOURCE
RESP
DOM. SUB-DOMAINE
QUALITY QUANTITY/REGULARITY

Prepare Tutorial Concept Provides a graphic representation of essential material to

Tut PD Prior to beginning of semester.
Map be covered in course
Prepare Tutorial Activity Details sequence of coverage to course substance across
SEMESTER PREPARATION

Tut PD Prior to beginning of semester.

Plan eight Units.
Details actual subjects and activities for each week/Unit
Prepare Tutorial Activity
Tut PD of the course. Includes assignments, worksheets and tests Prior to beginning of semester.
Units
to be used.
Develop Three Instructions and material for the assignments to be given
Tut PD Prior to beginning of semester.
Assignments to students in Weeks 3, 5 and 7.
Should contain introduction to the week's topic, links to
Develop Weekly Initiation
Tut PD extra materials or open-source resources prepared by Prior to beginning of semester.
Material
tutor, discussion questions and instructions for posting.
Upload Weekly Initiation
Tut US Upload all materials needed by students. Prior to beginning of semester.
Material and Assignments
Introduction/Welcome Tut PD Tutor orients online students to tutorial. During first session
Students post answers to week's discussion question and
Online discussion posting Std PD In every weekly session.
SESSION CONDUCT

comments on each other's postings.

Tutor logs on every day to website, reacts when
Participation, supervision No expectations specified
Tut US appropriate to student postings and reminds those who
and reaction about reaction or interaction.
are off topic or fail to post

Monitor conduct of online Vice-Dean or Online Tutorial Coordinator monitor online At least once during semester
UT US
tutorial conduct of tutorial to verify quality and compliance but no expectations specified.
Tutor reads and grades assignments completed by each
Grading assignments When completed
student in weeks 3, 5 and 7, while noting results.

75
Table 4.5. – continued

EXPECTATIONS AND SPECIFICATIONS

SOURCE
RESP
DOM. SUB-DOMAINE
QUALITY QUANTITY/REGULARITY

Grading discussion Tutor reads and grades discussion postings by each At the end of each weekly
Tut PD
postings student, while noting results. session.
Providing feedback and
Tut PD No expectations in this regard No expectations
interim grades to students
PEDAGOGY

Essential pedagogical
TUTOR

principles to be practiced Tut US No particular principles or procedures are advised. No norms established
by tutors.

Total and average session

Tut PD Tutors record the final grade of the tutorial After each session
discussion grades
SEMESTER WRAP-UP

Report tutorial grades to

Tut PD Tutors informs the final grade of the tutorial to students At end of semester
students
Report tutorial grades to
Tut PD Tutor report the final grades At end of semester
Examination Center
Report tutorial to the Vice
Should contain how tutor conduct the tutorial in the
Dean of Student Support Tut CS At end of semester
session
Affairs

76
Table 4.6. Fidelity of program implementation

OBSERVATION OF COMPLIANCE
DOM SUB-DOMAINE
RTG VAR COMMENTARY

Concept Map is usually part of the course plan developed by course manager
Prepare Tutorial Concept Map Low Low
(who usually also the tutor of the course) and not officially used in the tutorial
CLASS PREPARATION

Prepare Tutorial Activity Plan High Low All tutors must prepare the Tutorial Activity Plan

None of the tutors upload the Tutorial Activity Unit. Some tutors say, that
Prepare Tutorial Activity Units Low Low students can see the plan in Tutorial Activity Plan because TAU is the break
down from TAP
Develop Three Assignments High Low All the tutors develop the assignments

Develop Weekly Initiation All tutors prepare initiation materials, discussion questions and the instructional
High Low
Material posting. The extra materials should be prepared as needed (optional).
Upload Weekly Initiation
High Low Initiation material is available in all tutorial sites.
Material and Assignments
Welcome statement is basically the tutor welcoming the students in it tutorial.
While introduction is a brief introduction from the tutor to introduce him/herself
SESSION CONDUCT

Introduction/Welcome Low Low with name, educational background, and professional background. But only one
tutor officially introduces him/herself assuming that the tutor and all students’
brief information is available in the website where the tutor and students can see.

In general only about 40-50% students actively participate in the discussion by

Medi Medi posting opinion or comment. Some students post very minimum posting but some
Online discussion posting
um um students post many comments, particularly if his/her post attracts other students to
comment.

77
Table 4.6. – continued

OBSERVATION OF COMPLIANCE
DOM SUB-DOMAINE
RTG VAR COMMENTARY

Eight out of ten tutors only provide remarks, highlight, or conclusion at the end of
Participation, supervision and the discussion. Three tutors involve in an interesting topic where many students
Low Low
reaction send responses. Half of the tutors give encouragement especially when the post
traffic is low.

The monitor is usually conducted in the middle of the tutorial session or when
there is an issue that has to be resolved. At the end of semester, monitoring will
Monitor conduct of online
Low Low be done to examine if the tutorials have been performed completely. The time for
tutorial
the monitoring however is not rigidly schedule. The Vice Dean or the Head of the
Program can access to all tutorial sites anytime needed.

There is no technical format how to do the grading of the assignments. The

Grading assignments Low Low assignments are usually in short essay and the tutors arbitrarily take the power in
grading

There is a rule in grading the student participation discussion: Passive

participation: accessing the website, only passively read the initiation materials,
Grading discussion postings High Low questions, and responses from tutor and other students (20%).
Active Participation: Actively participate by posting comments, responding to
questions and comments from tutor and other students (30%).

There is no rule the tutors should provide feedback. All tutors do not provide
Providing feedback and interim feedback on the assignments, particularly because the tutor workload is
Low Low
grades to students overwhelming. Three tutors explicitly said that they were not be able providing
feedback on the assignments because they had not enough time.
PEDAGOG

There is not yet training on pedagogical principle in online training for tutors. The
TUTOR

Essential pedagogical principles

Low Low University, however, has a plan to conduct this training for the tutors in the next
Y

to be practiced by tutors.
future very soon.

78
Table 4.6. – continued

OBSERVATION OF COMPLIANCE
DOM SUB-DOMAINE
RTG VAR COMMENTARY

All tutors record the final grade of the tutorial as part of the report at the end of
Calculate grades High Low
the session.
SESSION WRAP-UP

The grades are supposed to be informed to the students. Only one tutor reports the
Report grades to students Low Low
grades to the students. The other tutors said that they had not enough time

Report grades to Regional Center High Low All tutors report the final grades to Examination Center

Submit report to the Vice Dean of
Student Support Affairs
Low
Not all tutors are able to develop the report. The tutors said that to report the
Low student grades take a lot of time and they have no time to develop the report.
Some tutors however send a brief report to the Vice Dean.

LEGENDS: RTG= Rating, VAR= Variation

79
session the tutor might create a different strategy for discussion by making small groups in order

to encourage more effective interaction. The levels of variation are defined as follows:

Low = 70%-100% tutors have similar formats, methods, or strategies

Medium = 40%-70% tutors have similar formats, methods, or strategies

High = 40% or less tutors have similar formats, methods, or strategies

In neither case (assessment of fidelity or variability of implementation) does the rating

refer to the quality or effectiveness of what was done, only to its correspondence to established

practice and to the degree of uniformity among tutor practices.

It should be noted, though, that the task of implementation is confronted with several

unpredictable elements. Chief among these is changes in the number of student enrollees. There

is no regular pattern explaining the increase or decrease in the number of participants each year.

Sudden change in the number of participants sometimes becomes a problem, particularly when

those numbers rise sharply and extra tutors are not available.

Resources

Although UT wishes to provide the best learning service to the students, the network

relies on the public services available. During interviews with student samples, three students

said that sometimes they had problem accessing UT’s website including the site would not load

or access was very slow. The students did not know if the internet connection problem was in

their area or in UT website. One of the three students admits that in a district in Central

Kalimantan, the internet connection is not very reliable. All the students interviewed however

realized that sometimes the server at the University was down for the maintenance and the

students understood the situation because the schedule was always informed by UT.

80
 According to one of the technical administrators offering the online tutorial for students

who live all over Indonesia, UT has to face some obstacles regarding to the general conditions in

Indonesia, most importantly, the uneven establishment of ICT infrastructure. The archipelagic

geography of Indonesian area and uneven distribution of population have made it difficult to

install ICT infrastructure equally. As the result, some students may live in an area with good

internet connection and others may not. Secondly, UT students have various skill levels using

computers and the internet. UT students come from extensive variations social and economic

background. Some students have advances skills in ICT but others may be just a beginner. In

interviews, most of the students are acquainted with using the internet for their work at the office

or in their personal lives. However, one student said that she learned how to use internet when

she enrolled as a student the first time. She learned to use the internet after she got information

about the online tutorial, thus, she accessed the internet for the very first time when she joined

the online tutorial. The third obstacle to providing good service for e-learning at UT is the high

cost of hardware investment and online system development. As an educational institution UT

does not have an unlimited budget in its operation because the primary financial sources are from

the low student fee and a government subsidy.

Semester Preparation

Instructional material. In the regular process of the online tutorial conducted every

semester, no specific guide is used. But the basic rule of conducting the tutorial is still based on

the Implementation Guide of 2002 although the guide is not officially distributed and used any

longer. This is because the current online tutorial situation changed drastically compared around

the year of 2002. The tutors have made some adjustment in facilitating tutorials. Some

modifications have also been made to adjust with the current situation in terms of number of

81
participants, number of courses offered with a tutorial, and the number of tutors. For example,

the 300 maximum students in one class tutorial is an adjustment for the dramatic increase of

students who are interested in using the online tutorial. The suggested maximum number of four

classes managed by one tutor is also an anticipation of the increase load of tutors.

The program platform of the online tutorial also changed, to Moodle in 2002. The

performance and the features of Moodle are different from the previous platforms used.

With the quick development of ICT and extensive student enrollment into the program,

currently a team has been altering the model of the online tutorial in terms of strategy and

method of the implementation of the program. However, there is no result or product yet that can

be reviewed to add the information to this research.

Instructor/tutor. The instructors/tutors of online tutorial are academic staff who work at

the head office or at the regional offices. Small number of the tutors works at the regional

offices. In the early years of online tutoring, all the tutors were from the head office in Jakarta,

but the growing numbers of students who participate in the online tutorial also increased the

number of classes. When the access grew, the number of students interested in the online tutorial

also grew. For courses which naturally have large number of students there is also a need for

many tutors to handle the number of classes of tutorial. When adequate numbers of tutors cannot

be found at the head office, the Faculty recruits academic staff who work at the regional offices.

Tutor training. All instructors/tutors have to pass a basic tutor training session before

they become a tutor. This training is basic for any kind of tutor at UT. The training is actually the

training provided to face-to-face tutors where the trainees learn instructional delivery for

classroom tutoring. In this training, however, the trainees learn about the model of the tutorial,

developing tutorial planning, conducting tutorial classes, and developing tasks, assessment, and

82
grading. The outputs of the training probably cannot entirely be implemented for the online

tutorials but the skills will be very useful for conducting online tutorials with some adjustments.

Additional e-tutoring training is also compulsory for the tutor of online tutorials, besides

the basic tutor training. This training is to introduce the features of the Moodle program of the

online tutorial system. This technical training is to familiarize tutors with the operation of the

program including uploading materials, importing open source materials, posting comments, and

other activities related to the online tutorial. Other than that, in this training, the tutors also learn

how to develop initiation question and material for discussion and assignment.

Six interviewed tutors mention that they all have attended and passed the two kinds of

tutorial. They also indicate that every semester before the online tutorial session starts, the

Faculty members have a regular meeting to determine if there is a new policy or strategy

regarding the online tutorial. Technical administrators from the Computer Center also attend the

meeting for tutors who want to refresh their skill on operating the computer program of Moodle.

Tutor workload. In the early years of online tutorial program, only small number of

courses offered the online tutorial program. Besides the program was not as popular yet, the

network and use of the internet had not been as extensive as it is today. At that time, a tutor

usually had to only manage one class with a small number of students. Even sometimes one

course had two tutors. With the rapid development of ICT infrastructure, the internet is becoming

more popular and accessible in Indonesia, as well as the online tutorial at UT among its students.

The number of enrollments in every online tutorial course has also been growing significantly

from semester to semester. Although the numbers of online tutorial enrollees vary from course

to course, for a course with large number of students has to offer about 10 classes of the online

tutorial. Consequently, this also demands plenty of relevant tutors with the course.

83
 According to eMarketer (2012), the number of internet users in Indonesia has been

increasing very significant from year to year. In 2012, internet users in Indonesia grew from 40

million in 2011 to 59.6 million or 24 percent of total population. The growth in the number of

internet users can be seen in Table 4.7. The growing number of students who are interested in the

online tutorial at FISIP is likely lined up with the increase number of Internet users, portrayed in

Table 4.8. Although the data only show the numbers of enrollees, and do not show the real

numbers of students, the significant increase can be seen from semester to semester.

Table 4.7: Prediction of Indonesia internet users 2011-2016

% of
Year Number of user (million)
population
2011 40.0 17.5
2012 59.6 24.0
2013 72.7 29.0
2014 83.6 33.0
2015 93.4 36.5
2016 102.8 39.8

Table 4.8: The growth number of enrollees to the online tutorial at FISIP

Year/Semester Nb. of Enrollees

2010.1 24,825
2010.2 32,798
2011.1 35,876
2011.2 57,922
2012.1 62,657
2012.2 82,589
2013.1 95,237

84
 Unfortunately the growth of the popularity of the online tutorial has not been followed by

equal increase in the number of tutors. The data from FISIP, in Table 4.9, shows that the number

of academic staff which is also tutors in the Faculty is basically constant. Furthermore, from

these numbers, some of them are off duty for various reasons such as pursuing advanced

education. That means not all staff can be recruited as a tutor. Anticipating the increase class

number of the tutorial, the FISIP recruits academic staff who work at the regional offices. The

additional tutors, however, are not proportional with the increase of class number of the tutorial.

Table 4.9: Data of academic staff/tutor at FISIP from 2008-2012

Year Number of academic staff

2008 119
2009 121
2010 123
2011 123
2012 124

The maximum participants in a class of online tutorial are 300 students, but the students

who actually active in its activities are around 30-50 percent. Although the number of active

student is less than half but the tutors still feel that the class is relatively big. The size of the class

influences the effectiveness of the class management. More than the class size, the number of

class be managed by a tutor also brings some concerns. Although there is no policy about the

number of classes that should be managed by a tutor, four classes are the common understanding

among the tutors. The fast growing number of students interested in the online tutorial, however,

has made the number of classes also increase significantly.

85
 Time allocation. Unlike an academic staff at the conventional university who do not

have a rigid office schedule, the academic staff at UT must be present at the office for a regular

40 hour week. Their main tasks are not only tutoring but also conducting research and providing

community service. They also have a responsibility for writing course material, developing

multimedia materials, writing examination material, and other additional tasks. Many staff also

hold other positions like dean, vice dean, head of center, or other positions. Tutoring, therefore,

is only a part of their main responsibilities.

If three-credit-hour course is equal to three hours teaching every week, tutoring means

the tutor has to spend three hours uploading material, moderating discussion, reading student

assignments, giving feedback, and scoring. If a tutor has four classes to manage, the tutor has to

devote 12 hours of their working hours for tutoring. In reality, time for tutoring cannot be rigidly

calculated since the tutorial is happening along the week. Students’ posts and assignments come

any time during the week and the tutor has to monitor the progress via the internet. In each

tutorial, besides the discussions, the students also have to complete three short essay

assignments. Reading, correcting, and scoring the assignments also takes time.

Tutoring four or more large classes is very time consuming. With many other

responsibilities most of the tutors say that they have not enough time to be “good tutor”. One

tutor says that she wants to be able to respond all student posts individually and give feedback to

all assignments as she did before when the tutoring work load is not as heavy as tutoring

recently. Now, to read all posts thoroughly and provide feedback to the discussion she has to

spend one whole working day. She frequently has to continue working on the tutorial at home, as

she says, “I often continue working on the tutorial at night at home after my kids have gone to

bed”.

86
 Scoring is another task that consumes time. At least there are two kinds of tasks i.e.

scoring for assignments and student participations in the discussions. According to the tutors, to

grade the students’ participation in the discussion, the tutors not only have to read the posts but

also have to check in which week each student posted the required opinion or comment. Scoring

the assignments, the tutors have to read comprehensively all the submitted assignment and judge

for scoring. Not to mention if the tutors have to give feedback. In the interview, all the tutors

admit that they spend a lot of time fulfilling their tutorial duties.

Tutorial Process

Student affairs. In the process of joining the online tutorial, students do not specifically

register with the learning service. The online tutorial is part of the integrated student online

service called UT Online. The services of UT Online include basic service, independent learning

support service, and online tutorial. The basic service mostly offers information to students on

general information and academic administrative information. The independent learning support

service aims to provide services that can assist student in their independent learning such as a

digital library service, independent online exercises, an online bookstore, and a supplemental

course material service.

To access all the online student services, the student does not have to register individually

for the services but signs up to UT Online as a member. In the process of registration, the student

is required activate his/her student identification and submit an e-mail address. Once the student

account is activated, the account will be kept in the UT database. When a student registers for

courses every semester, the student is automatically registered in an online tutorial for the

courses. The student can only participate in the online tutorial of his/her registered courses.

87
 Since the maximum number of students in the online tutorial is 300, the automatic

registration of the online tutorial will put every 300 students in a group. If, for example, a course

is registered by 700 students who have active account in UT database, the first 300 students will

be in first group, the second 300 students will be in second group, and the last 100 students will

be in the third group. The online tutorial begins on week 2 or 3 of the semester. Registration for

the online tutorial can be done until the end of second week of the online tutorial. This means

that although the online tutorial has already started, when new students register for UT Online

and activate their account, the students can still join the tutorial if they want to. As a

consequence, at the third week of online tutorial time there is the possibility that a new group is

added to the tutor who responsible for the course.

Participation in the online tutorial is voluntary. This means the students have the option

to take advantage of the service. Although the maximum number of students in one group is 300

students, normally only around 30-50 percent of the students registered participate in the online

tutorial. Participation is calculated from the number of students who actively participate in the

discussion. Low student participation in the online tutorial may be a consequence of automatic

registration. Some students probably do not realize that they are registered for the online tutorial,

and some others may know that they are registered but they are not interested in participating.

Table 4.10 below presents data on the numbers and percentage of active students in each of the

sample courses.

88
Table 4.10. Number and percentage of students active in online tutorial

Number of students Number of active Percentage of active

Course
enrolled students students
C1 276 116 42%
C2 270 126 47%
C3 228 112 49%
C4 276 123 45%
C5 278 135 49%
C6 168 78 46%
C7 245 90 37%
C8 276 126 46%
C9 149 67 45%
C10 275 84 31%

Student orientation. Orientation to online learning, together with other distance learning

matters, has been conducting regularly by UT in the orientation session at regional offices at the

beginning of every semester. This orientation is for new students who are not yet familiar with

the distance education system. In this orientation students are introduced to the learning methods

and facilities that can help students in their study. UT Online is one of UT programs that are very

important to be introduced in the orientation program because this program distinguishes

distance education from conventional methods of education that students should be familiar with.

More importantly, this program requires students to have computer skills and internet access

which may be new to some students.

This orientation program, which is held in the regional offices, however, will not cover

all new students since not all students are able to physically attend. Anticipating that some

students would not be able to attend orientation, UT put the information about the online tutorial

in its catalog and on its website. There is general information about the UT Online program on

89
the website that provides information about all online programs offered, including the online

tutorial. Also, there is specific information about the online tutorial which is actually a guide

how to activate the account of the online tutorial. In more detail, this guide is basically a

technical manual directing students how to operate the features of the online tutorial program.

On the sites of ten online tutorials observed, all tutors start the session with a welcoming

statement to their students. Other than the greeting, eight tutors provide information about the

activity in the discussions and completion of the assignments that will determine the final student

score of the tutorial and its contribution to the final course grade. Three tutors offer encouraging

words for their students to keep active in all of the tutorial activities.

Each course of the online tutorial, however, does not offer further information for student

orientation. Most of the tutors do not realize that some students might need further direction on

how to participate in the online tutorial, because the tutors already have habit of tutoring. The

tutors assume that the website of UT has provided enough information about the online tutorial.

Eight of the ten interviewed students say that they used with the internet in their daily

activities and in their workplace. These eight students assert that they never had difficulty in

completing assignments and accessing the program of the online tutorial. For the program, it can

be because not all features in the online tutorial program are used and the features are friendly

enough for the people who are familiar with internet function and operation. There were

problems with accessing the UT website or internet connection but not with the features in the

online program.

The other two students admit that they are not very familiar with using the internet. One

of them was introduced to the internet for the first time when she became a student at UT. When

she has a problem in operating the program and completing the assignments she “asks help from

90
a friend”, (a colleague in her workplace who is also a student in UT and has experience on the

online tutorial). Another student from Bali who is also inexperienced with the online tutorial says

that she has a regular meeting with her colleagues where she can ask for help.

Online tutorial guidelines. It is very important to create a climate that makes the

students feel comfortable in the online learning environment. Students new to online learning

may initially find this kind of learning disorienting without a physical classroom space and

guidance from the physical presence of a teacher. Other students may feel anxiety using internet

technology. Some students may at first misperceive that learning online is easier than classroom-

based learning. To ease the disorientation and confusion good guidelines for all activities are

needed.

From the observation of ten courses, by reviewing complete records of the online tutorial

in second semester of 2012, very few of the courses provide clear guideline in its activities. In a

discussion, for example, there is no guide about time frame, required participation, how feedback

will be given, or other information on participating in the discussion. There is a brief guideline

for completing assignments found in two course tutorials, and the other eight courses only

provide questions without further instruction about how the assignments should be completed. It

will be easier for the students if they have more detail about the format, length, structure of

writing, due date, and other necessary information. Since the students have no experience on

academic writing, a guide with writing tips will help them to accomplish the assignments as well

as to learn how to develop good academic writing.

Online discussion and interaction. Although there is an understanding that one of the

objectives of online tutorial is to give a chance for students to interact with the instructor, in the

written document the Guide of Online Tutorial Implementation 2002, interaction was never

91
clearly defined. In the Guide, it is stated that the online tutorial is an alternative tutorial to give a

chance for students who have access to the internet to have the experience of a classroom

situation via the internet. UT realizes that many students do not have the opportunity to join a

face-to-face tutorial in the area where they live, there are not enough students to create group for

a face-to-face tutorial and/or there is no qualified tutor found in the area to facilitate the

classroom tutorial. By utilizing internet technology to support student learning, UT gives a

chance for the students to communicate with their instructor.

The understanding of interaction in the online tutorial thus depends on the perception of

the tutors. Some tutors say that they want to interact with students by giving responses to each

post in a discussion and some other say that less intimate interaction is enough such as giving

remarks, highlights, or a conclusion at the end of the discussion. The tutors who want to provide

individual responses think that the students will be more satisfied in the online tutorial if they are

able to respond posts or questions separately. This tutor understands the students’ need for

interaction because the students want to experience learning similar to the face-to-face

classroom. The tutor remembered when she only had one small class of tutorial she had very

intensive interaction with her students. She also provided an email address for student who

wanted interact with her if any student had any other inquiry related to their study. She indicated

that even after the online tutorial session had ended, some students still contacted her for advice.

On the other hand, the tutors who prefer to give remarks or highlights at the end of discussion

think that there is no definite answer in social science. Any opinion from different point of view

is accepted as long as the student’s opinion sticks to the topic. These tutors argue that

highlighting is more appropriate, particularly, for those who have different opinions.

92
 Related to interaction as mentioned in the Guide 2002, a tutor who is also a member of a

team which initially developed the online tutorial, says that at the beginning the objective of the

online tutorial was not primarily for interaction but “… to give an opportunity for students

participating in the tutorial because they had no chance to attend a face-to-face tutorial”. The aim

was to allow students to “feel in school”. In the initial year of the online tutorial program, UT

was really concerned about the possibility that most of the students felt isolated by reading the

main printed material independently. The most popular student learning support was provided in

a face-to-face tutorial. Students who could afford this learning support were limited to the

students who live around the tutorial locations. Students from everywhere could create a 20-

student group and to hire a tutor for a face-to-face tutorial. But in reality, it was very difficult for

students to find other students taking the same course in his/her area for the tutorial group. For

those unfortunate students, UT tried creating learning support media to reach as many students as

possible.

The tutor who is also one of the online tutorial initiators says that at the beginning the

tutorial was a forum in the form of mailing list of students. In this forum the students could

interact with anyone in the list including the administrators and they could discuss about any

subject related to their study. When the number of participants in the forum grew and the internet

was more accessible, UT developed the preliminary online tutorial program in addition to the

forum. Currently, the forum is called the Online Forum and the tutorial is the Online Tutorial.

The tutor emphasizes that the basic idea of the online tutorial is to give students

participating in the tutorial to have a chance to interact with the class. The interaction, however,

does not always occur between student and instructor, but also between student and student and

between student and the course content. The students interact with each other in the discussions.

93
Interaction between the students and the instructor happen occasionally when the instructor

responds to discussions about an interesting topic, gives remarks or highlights, and responds to

questions or inquiries. In order to participate in the discussion of topics taken from the main

course printed material, students are required to read the related chapter and additional materials.

This means that the online tutorial encourages students to interact with the content of the course.

Related to the objective of the online tutorial which actually does not emphasize the

interaction between the students and the instructor, one tutor who is also a member of initiators

of the online tutorial reminds that online learning formats and methods are adopted from the

Western culture of learning. In Western culture, the opinion of students and of teacher or

instructor may be considered as equal. Students can say their opinion without hesitation. On the

contrary, Indonesian students used to a teacher-centered culture of learning where the teacher’s

opinion is considered as the right answer by the students, would be considered as impolite. The

Indonesian teacher-centered learning culture positively influences student’s attitude. They are

not sure whether the tutor agrees with their opinion and conformity to the tutor is important to

them.

Among students interviewed, a student says that he completes all required assignments

but never participates in the discussion sessions; he never posts any comment or opinion. This

student says that he does not want to post or send any opinion because he only needs to read the

other student’s posts which actually give him variation in his study rather than just reading the

main printed material and studying by himself. In this tutorial this student does not wish to

interact with the instructor. This student behavior also supports the assumption that the reason

for students to participate in the online tutorial is not always for interaction.

94
 Another interesting support to the assumption that interaction is not the main purpose of

students joining the online tutorial is that in an online tutorial that only has a small number of

students, in the seventh and eighth week there are no student posts in response to the discussion

question. When a student in this tutorial was interviewed the student said that he did not want to

post any opinion because there is no post from other students. He said that he did not want to

interact “individually” with the tutor if there were no other posts. In this case, the student did not

want to have intensive individual interaction with the tutor even if he had an opportunity to do it

since the tutor usually gave responses to the student posts every week.

Table 4.11. Number of students who do and do not participate in the online tutorial activities

Number of Students
Course
Participate in Submitting
Total Access Only Never Access
Discussion Assignment

C1 276 116 42% 87 32% 81 29% 79 29%

C2 270 126 47% 74 27% 67 25% 77 29%

C3 228 112 49% 87 38% 51 22% 65 29%

C4 276 123 45% 90 33% 60 22% 93 34%

C5 278 135 49% 90 32% 53 19% 90 32%

C6 168 78 46% 69 41% 25 15% 65 39%

C7 245 90 37% 72 29% 75 31% 80 33%

C8 276 126 46% 68 25% 53 19% 97 35%

C9 149 67 45% 53 36% 42 28% 40 27%

C10 275 84 31% 80 29% 90 33% 101 37%

95
 Besides the information from student interviews, data from the observation also show

that some students passively access the sites, as recorded in the Table 4.11 above 15-33% of

students from the 10 online tutorials who actually access the sites never participated to any

activities. These numbers are calculated from the total student participants in an online tutorial

reduced by the number of students who participated in the discussion and the number of students

who never accessed the site of tutorial. In general the number of students who participate in the

tutorial is bigger than the number of students who submit the assignments. It is a simplified

assumption that the students who submit the assignment are the students also the students who

participate in the discussion.

It needs to be mentioned that interactions between student and student, student and

faculty, and student and administrator not only occur in the online tutorial activity, but also in

other programs or forums for the students. Besides the online tutorial, UT offers Online Forum

where the students can interact with other students, with faculty, and administrator. Because of

the large number of students accessing the site, this forum is separated into each program where

the student is registered. In the forum students can post questions, inquiries, problems,

announcements, news, or any other matters that are relevant to their study at UT apart from the

content of course material. Other students can respond to the posts or discuss them in this forum.

Inquiries to faculty are answered by the Course Manager, Head of Program, or Vice Dean of

Student Support Affair. For the administrative matter, an administrator will be available to assist

the students.

96
 CHAPTER FIVE

THE RESULTS OF THE ONLINE TUTORIAL PROGRAM

Introduction

In the early rounds of implementation of the online tutorial, UT was concerned about the

low numbers of students who participated in the program – largely due to the limited number of

internet users among UT students. Although UT believed that the infrastructure of information

and communication technology (ICT) would expand to more areas in Indonesia most UT

students live in rural and even remote areas. Years after that first development, the ICT

infrastructure has in fact reached a much broader area of Indonesia. Large numbers of students

now enjoy access to the internet for their daily life and for their study.

In the last three years the number of students who have been interested in the online

tutorial has increased dramatically. The significant increase in the number of student participants

in the program unfortunately has not aligned with the growing number of tutors. UT has also

started to hire tutors from other universities because of the lack of qualified tutors. However, the

policy of hiring outside tutors will take a while to take effect since the hired tutors need to

become familiar with the distance education system.

In this time of transition, current UT tutors have to manage many large classes. The tutors

feel that they cannot perform well in their work. They cannot provide proper feedback and do not

have enough time to maintain interaction with their students during the tutorial. On the other

hand, although the students express their dissatisfaction with the program related to the grading

transparency and the lack of interaction with their tutor, the students say that they need the

service for their study and they think that the service assists them in learning.

It is important therefore to begin examining whether the online tutorial service does in

fact facilitate student learning, at least insofar as their progress is measured in the final course

97
examinations. The quantitative analysis portion of this research was devoted to investigating

whether the mean of the final exam scores of students who participate in the online tutorial is

significantly different from the mean scores of non-participating students.

Analysis of Final Course Examination Results

T-test analysis was conducted to examine the results in the ten sample FISIP courses in

this study, randomly selected from among the hundreds for which online tutorials are available.

The null hypothesis states that the online tutorial has no effect on student performance on the

final exam and that there is no significant difference between the group means. The alternative

hypothesis affirms that there is significant difference between the mean scores of the two groups.

H0: u1-u2 = 0

H1: u1-u2 # 0

u1: the mean score of final examination of group student participants in the online tutorial

u2: the mean score of final examination of group student non-participants in online
tutorial

Descriptive Data

Table 5.1 below presents essential descriptive data concerning test results in the ten

sample courses. The real numbers of students who registered for the courses vary from 93 all the

way up to 3853 students. However, the sample chosen from each course is much smaller, as it is

drawn solely from students who take the online tutorial, and who represent overall 2128/17552

or about 12% of those enrolled. Moreover, the sample for this study was further limited by

exclusion of those who take BOTH the face-to-face and online tutorial in order to focus on

students getting their learning support only via the online connection and to avoid the bias on

examination courses that their double tutorial experience might create. That reduced the effective

sample by another 200 students down to 1,928, or about 11% of total enrollment in the ten

98
Table 5.1. Descriptive data on sample courses

Actual Nb. Online + T-test

Course Online Tut. F2F Tutorial
of Students F2F Tutorial Sample

1 2,890 247 684 27 220

2 1,299 219 171 8 211

3 1,567 194 271 20 174

4 3,584 173 866 16 157

5 3,853 482 779 60 422

6 3,038 411 786 41 370

7 445 129 98 9 120

8 321 117 29 9 108

9 462 144 79 10 134

10 93 12 0 0 12

TOTAL 17,552 2,128 3,763 200 1,928

courses. Sample sizes per course averaged therefore about 193 students but in fact varied from

12 to 442.

The readers should also remember that, as discussed at the end of Chapter 4, some of the

students enrolled in the online tutorial do not participate fully. The subjects of the study were not

selected from the students who participate in all of the activities of the tutorial, since scores

cannot be correlated with level of completion of online course duties. Only students identified as

having an online tutorial score are included as participants.

Another important point to note about the samples concerns the t-test results and the

number of non-participant students in the sample. As is evident from the data in Table 5.1 above,

the actual number of students enrolled in any given course who do not participate in the online

99
tutorial is much greater than the number of online tutorial participants included in the study

sample. Overall, in excess of 11,000 students who were registered in the ten courses did not

participate in any form of tutorial, whereas the number of online tutorial students in the student

sample numbered just under 2000. The size of the t-test sample, which was randomly selected, is

equal to this number.

Table 5.2. Descriptive data on ten sample courses final exam results

Course Group N Mean Std. Deviation Std. Error Mean

Tutorial 220 51.7272 11.97908 .80763

1
Non-tutorial 213 48.5238 13.85671 .94945
Tutorial 211 53.5640 9.99235 .68790
2
Non-tutorial 217 48.2304 11.17878 .75887
Tutorial 174 42.1464 9.36428 .70990
3
Non-tutorial 184 42.0527 10.88497 .80245
Tutorial 157 43.2046 11.37645 .90794
4
Non-tutorial 178 39.5009 11.09386 .83152
Tutorial 422 34.6919 11.59879 .56462
5
Non-tutorial 447 32.5011 11.96705 .56602
Tutorial 370 44.4108 13.19867 .68617
6
Non-tutorial 365 39.4904 12.36768 .64735
Tutorial 120 42.2167 11.21462 1.02375
7
Non-tutorial 122 43.0000 13.40347 1.21349
Tutorial 108 48.0556 10.07062 .96905
8
Non-tutorial 113 48.0354 12.31428 1.15843
Tutorial 134 43.4627 9.71214 .83900
9
Non-tutorial 138 42.4058 12.69556 1.08072
Tutorial 12 37.7767 12.73995 3.67771
10
Non-tutorial 22 30.6059 11.84751 2.52590

100
 Data concerning the sample of online tutorial participants and non-participants from each

course whose scores were compared by t-test is displayed in Table 5.2 above. It was decided to

include roughly equal numbers of online tutorial participants and non-participants in each of the

ten courses in order to ensure approximately equal standard error means and avoid statistical

inaccuracy in sampling and testing.

The overall mean scores of student participants and non-participants are not very high.

The mean scores of the student participants among the 10 courses are between 34.69 and 53.64

on a 100-point scale, while non-participants’ scores fall between 30.61 and 48.52. For both

participants and non-participants, the statistics show standard deviations ranging from 9.36 to

13.20.

T-test Results

In the next stage of analysis, t-tests were conducted to determine the significance of

differences between the average final course exam scores of online tutorial participants and the

average scores of students who did not participate in the online tutorial. The results are presented

in Table 5.3 below.

The results of Levine’s Test for Equality of Variances displayed there indicate that the

significance levels (Sig.) of the two groups are mostly greater than .05, meaning that group

variances of most courses are equal. Only Course 8 and 9 display unequal variances. The t-test

results for the ten courses indicate that five of the ten online tutorials (Courses 1, 2, 4, 5, and 6)

are characterized by significant differences between the means of the online tutorial participants

and those non-participants with t-values ranging from 2.58 to 5.21, whereas for a sixth course

(Course 10) results are just short of the 0.10 significance level with a t-value of about 1.64.

These statistics suggest that the online tutorials of five or six courses have a significant effect on

101
student grades for the final exam. In the four other online tutorials (Course 3, 7, 8 and 9),

however, t-test results are far from significant and t-values are minimal, ranging -0.49 and 0.77.

Table 5.3. Results of independent samples t-test analysis

Levene's Test for

Equality of t-test for Equality of Means
Variances

95% Confidence
Assumptions

Interval of the
Sig. (2- Mean Std. Error
F Sig. T Df Difference
Course

tailed) Difference Difference

Lower Upper

EVA 2.438 .119 2.576 431 .010 3.20343 1.24356 .75922 5.64763
1
EVNA 2.570 417.958 .011 3.20343 1.24648 .75327 5.65358

EVA 2.535 .112 5.199 426 .000 5.33357 1.02586 3.31719 7.34994
2
EVNA 5.207 423.030 .000 5.33357 1.02425 3.32032 7.34682

EVA 1.001 .318 .087 356 .931 .09372 1.07589 -2.02218 2.20961
3
EVNA .087 352.890 .930 .09372 1.07140 -2.01341 2.20084

EVA .012 .913 3.013 333 .003 3.70375 1.22923 1.28573 6.12178
4
EVNA 3.008 325.571 .003 3.70375 1.23117 1.28170 6.12580

EVA .068 .795 2.738 867 .006 2.19082 .80021 .62026 3.76139
5
EVNA 2.740 866.398 .006 2.19082 .79949 .62167 3.75998

EVA 1.249 .264 5.214 733 .000 4.92040 .94376 3.06761 6.77319
6
EVNA 5.216 731.073 .000 4.92040 .94334 3.06842 6.77238

EVA 2.088 .150 -.493 240 .623 -.78333 1.58998 -3.91542 2.34876
7
EVNA -.493 234.003 .622 -.78333 1.58765 -3.91125 2.34458

EVA 4.375 .038 .013 219 .989 .02016 1.51713 -2.96989 3.01020
8
EVNA .013 213.936 .989 .02016 1.51030 -2.95682 2.99714

EVA 5.551 .019 .770 270 .442 1.05689 1.37346 -1.64716 3.76094
9
EVNA .772 256.085 .441 1.05689 1.36816 -1.63740 3.75118

102
Table 5.3. – continued

Levene's Test for

Equality of t-test for Equality of Means
Variances

95% Confidence
Assumptions

Interval of the
Sig. (2- Mean Std. Error
F Sig. T Df Difference
Course

tailed) Difference Difference

Lower Upper

EVA .114 .738 1.643 32 .110 7.17076 4.36446 -1.71937 16.06088

10
EVNA 1.607 21.338 .123 7.17076 4.46158 -2.09866 16.44017
Legend: EVA: Equal Variance Assumed, EVNA: Equal Variance Not Assumed

This suggests that participation in the online tutorial for these four courses had no effect on

improving students’ scores on the final course examination.

The third research question of the dissertation concerns the effect of participation in the

online tutorial on success in final course exams. To answer the question, the t-test statistical

results must be interpreted to represent the degree or magnitude of mean difference between the

final exam scores of participants and non-participants. As a next step, therefore, I examine the

effect size of t-test results for the courses in which there was a significant mean difference. To

measure the effect size, this research uses Cohen’s d technique. Possible degrees of mean

difference are categorized as follows:

d = 0 - 0.2: small effect

d = 0.2 – 0.5: medium effect

d > 0.5: large effect

Results of this analysis are portrayed in Table 5.4 below. Examination of Cohen’s d

effect size results for the five courses characterized by a significant difference of group means

(and excluding Course 10, where results did not quite attain the 0.10 significance threshold)

103
indicate that one online tutorial program has a large difference mean, three have medium-sized

difference means and one has a small difference mean.

It is interesting to note that in the mean differences between the scores of the online

tutorial participants and the non-participants half of the numbers are significant and the other half

are not significant. With only half of total sample courses show significance means difference, it

is appealing to learn the statistical result of collective sample scores of the ten courses in

aggregate scores t-test. The aggregate t-test score will show whether the participation in the

online tutorial in general has an effect on the final exam score. Below are the results of the

aggregate t-test of the ten courses.

Table 5.4. Summary t-test results and effect sizes across ten courses

Means Cohen’s d
Course Results Interpretation
difference effect size

1 t(431) = 2.58, p= .01 significant 0.248 medium

2 t(426) = 5.2, p= 0 significant 0.5039 large

3 t(356) =.87, p= .93 not significant - -

4 t(333) = 3.01, p= 0 significant 0.3297 medium

5 t(867) = 2.74, p= .01 significant 0.1859 small

6 t(733) = 5.214, p= 0 significant 0.3849 medium

7 t(240) = -.49, p= .62 not significant - -

8 t(219) = .013, p= .99 not significant - -

9 t(270) = .77, p= .44 not significant - -

10 t(32) = 1.61, p= .11 not significant - -

104
 Tables 5.5 and 5.6 below take up the question of testing aggregate differences between

participants and non-participants in the tutorial across all ten courses. The first presents

descriptive data on the aggregate t-test, whereas Table 5.6 offers the results of the analysis. In the

latter, it can be seen that the collective independent samples t-test gives evidence of a significant

difference between the mean of final exam scores of the online tutorial participants and of the

non-participants across all sample courses. This suggests that – considered overall --

participation in the online tutorial does improve final exam scores. However, the results of the

Cohen’s d effect size calculation on these figures presented in Table 5.7 show only a small

difference between the means (0.087).

Table 5.5. Descriptive data of aggregate t-test

Groups N Mean Std. Deviation Std. Error Mean

Aggregate Online tutorial 1928 43.7780 12.90955 .29401

Non-online 2031 41.4864 13.41908 .29776

tutorial

Table 5.6. Results of aggregate independent samples t-test

Levene's Test
for Equality
of Variances t-test for Equality of Means
Assumptions

95% Confidence
Sig. Interval of the Diff.
(2- Mean Std. Error
F Sig. t Df tailed) Difference Difference Lower Upper

EVA 2.580 .108 5.471 3957 .000 2.29158 .41887 1.47035 3.11281

EVNA 5.476 3956.295 .000 2.29158 .41845 1.47118 3.11198

Legend: EVA: Equal Varian Assumed, EVNA: Equal Varian Not Assumed

105
Table 5.7. Cohen’s d aggregate effect size

Course T-test result Mean difference Effect size Interpretation

Aggregate t(3957)=5.47, p=0 significant 0.087 small

Even though the overall t-test result shows a significant difference between the means,

the reason why the online tutorial for at least four of the ten courses apparently does not help the

students to improve their final exam scores requires further investigation. Some reasons why the

online tutorial might not seem to have a statistically significant effect on final exam scores

include the following:

• First, many students included in the group of online tutorial participants in the t-test

actually did not really fully participate in the tutorial. Many students have quite low

scores. It is suspected that they did not participate in the discussions or complete all the

assignments. They received a very low score for the online tutorial which indicates only

passively accessing the site. Their participation therefore did not really help them in

learning and preparing for the final exam.

• Second, some students admit that they actually did not have the main printed material for

the tutorial. They did not have enough money to buy the material or the delivery of the

material was late.

• Third, some tutors in the interview suspected that many students did not read the course

material because they rely on the online tutorial. A couple of students admitted that they

did not read the material because they thought that participation in the online tutorial was

enough to prepare for the final examination. This attitude was actually influenced by the

students’ learning culture of face-to-face education. They thought that the online tutorial

using distance education strategies was equal to lectures in a classroom-based setting. As

106
a result, the students received a very low score on the final exam because they were

actually not ready for the exam.

107
 CHAPTER SIX

INTERVIEWS AND INTERPRETATION

Introduction

The flexibility of course instructional methods used is one of three factors that define the

degree of transactional distance in educational delivery. Although transactional distance refers to

the psychological distance between instructor and students due to the separation of the two,

geographical distance between them can also increase the sense of separation. The degree of high

transactional distance that may results from a rigid instructional structure and strategy can be

reduced by the intensity of dialogue between the instructor and the students.

UT, which offers printed material as the main resource in its courses, realizes that

students need assistance in their independent study gained through contact with their instructor.

The online tutorial, which was initially offered as the alternative to face-to-face tutorials, is

currently considered the most promising learning support that can reach a broad base of students

all over Indonesia. Methods and strategy for conducting tutorials have been repeatedly modified

in order to adjust to the fast development of information and communication technology and to

rapidly increasing in student interest for the service. University management staff responsible for

the tutorial is still looking for the most appropriate model of the tutorial while considering a

variety of available support resources.

Data on the implementation of the online tutorial and on its results have been presented in

the preceding two chapters. Chapter 6 is devoted to interpretation of this record of achievement

and to the opinions and insights concerning major issues and underlying causal factors expressed

by key “local” stakeholders – both students and tutors. The section immediately following offers

a profile of the selected interviewees. The interpretive topics are then arranged in roughly the

same order as the preceding tabular and narrative presentation of the online tutorial model and its

108
implementation: first aspects of program “input,” then key dimensions of program process and

finally the issue of the results of the tutorial and their worth to stakeholders.

Profile of Interviewees

Student Participants

To develop a suitable model for an online tutorial definitely requires information from all

subjects involved in the implementation of the tutorial, the students, the tutors, and the

administrators. Ten students were selected for in-depth interviews. They were chosen from

different regional office areas, in order to avoid the bias of homogeneity of internet connection

quality. Although there is no detailed information about the quality of the internet connection in

each area, the variety of locales at least makes sure that the students do not come from the same

region. All the students hold a job. Some of them are able to access the tutorial website in their

spare time at the office, but most have to access the tutorial site from home.

The principal characteristics of the participants are presented in Table 6.1 below. As is

evident there, six are female and the other four are male. The students are in the first to sixth

semester with various levels of experience in operating the internet. These students were

interviewed to get information surrounding their experience participating in the online tutorial

such as the quality of their internet connection, the quality their of interaction with the tutor and

other students, the tutor’s feedback, general benefit of the service to their learning process, the

student workload, the weaknesses, and also their expectation for the future improvements.

Tutor Participants

Tutors who participated in the interview were selected from available tutors in the

Faculty of Social and Political Sciences (FISIP). Several tutors were emailed asking if they

wished to be involved with this study.

109
 The first six tutors who emailed back and consented to become the participants in the

study were accepted. One of online tutorials managed by each tutor also became a tutorial site

observed in the conduct of the study.

Characteristics of the tutor participants are presented in Table 6.2 below. Three were

females and three males. All of them are experienced tutors who have been involved in

facilitating tutorials for 4-9 years. In the second semester of 2012, the tutors managed four to

seven classes.

Table 6.1: Profile of student participants

Experience with Access to

Student Semester Sex UPBJJ (Reg. Office)
Internet Internet
Palangkaraya (Central
S1 3rd F Beginner Home and office
Kalimantan)
Pekanbaru (Riau,
S2 3rd F Advanced Home
Sumatra)
S3 3rd F Denpasar (Bali) Very beginner Home
Office and
S4 6th M Lampung (Sumatra) Advanced
mobile
S5 2rd F Bandung (West Java) Advanced Home
Makassar (South
S6 1st M Intermediate Office
Sulawesi)
S7 3rd F Surabaya (East Java) Intermediate Home
S8 4th F Bogor (West Java) Advanced Home
Mostly home and
S9 6th M Banten (Serang, Java) Advanced
office
Tanjung Pinang (Riau
S10 6th M Intermediate Home
Islands)

110
Table 6.2: Profile of tutor participants

Tutor Sex Tutoring experience Number of classes

T1 M 7 years 4
T2 F 9 years 7
T3 F 8 years 6
T4 M 4 years 5
T5 F 9 years 5
T6 M 9 years 4

Program Inputs

Quality of Internet connection

The principal issue of program resources raised in interviews concerned the quality of the

internet connection for the online tutorial. Although not all the students identified this as a

problem, the issue most often encountered during the online tutorial session was difficulty

accessing the UT website, slow connection times and technical skill deficits. Some students were

not quite sure about the problem, but from what they described, it is likely a problem with

internet connection. As student number 3 (S3) said, “Why does someone in the same group as

me have the discussion question but I do not?”

It seems impossible that students in the same group did not get the same discussion

question at the same time. Since the student is a very beginner user of the internet, the student

may not have enough skill in operating the internet or the internet connection is not good enough

for accessing the website. This student used the internet the first time when she joined UT as a

student. Luckily, this student had group mates at the same office whom she could ask for help.

The other problem is that the local connection of the internet is not good enough. As

mentioned by a student from Palangkaraya, Central Kalimantan (Student 1) who lives in a small

111
district surrounded by a forest that she often has connection problems. Kalimantan is the biggest

island in Indonesia but the population is small and lives sparsely which is probably why the

Internet service is not as good as in Java. Similar connection issues also faced by a student from

Tanjung Pinang (Student 10) where his area is basically in the province with many small islands

(Kepulauan Riau) which probably explains why the internet network is not very good.

The other issue accessing the internet and UT website is when UT does the maintenance.

For this matter, however, the students say that UT always informs students so that they know

when the server is down.

Since the online tutorial has somewhat rigid schedule, the problem of internet connection

seems be an issue for some students reducing their time to complete discussion posts or

assignments.

Program Process

Three topics related to the process dimension of the UT online tutorial were frequently

raised in the interviews: online interactions and discussion between students and tutors; issues of

feedback and grading; and student and tutor workloads. The fruit of those exchanges is reported

below.

Student-tutor Interaction in Discussion

One of the objectives the implementation of the online tutorial is to give a chance to

students to interact with the course instructor which in the online tutorial is represented by tutor.

From the observation of the tutorial website, in the discussion sessions for example, in general

tutors give no clear guidance on how the discussion should be conducted. After presenting the

introductory material and the discussion question, all the tutors instruct the students to discuss

the given topic or the question. However, there is no direction what the students should do, for

112
example, how many posts each student should send, which post the students should respond to,

what the tutor will do during the discussion, how the tutor will grade the participation in the

discussion, and so on. The absence of clear instructions makes most students submit the

minimum post, but some students post several comments responding to other students’ posts by

assumption that the more comments will give them better score/grade.

Unfortunately, the tutors only give general responses to all the post at the end of the discussion.

As the result, the students were dissatisfied because the tutor did not respond their posts

individually. The disappointments are expressed by some students. One said “I want the tutor

responds the individual question to develop individual closeness and to elevate student

confidence to perform in the tutorial.” Another said that “I want the tutor to respond to my

opinion in the discussion to let me know that I am giving the right argument. I need the tutor to

facilitate the discussion more intensely, at least by giving us suggestions or explanations when

the discussion is deadlocked.”

These arguments are mentioned by the students because they had an experience that in

the previous semesters where some tutors provided individual responses to the students posts in

the discussions. Although not all tutors gave individual responses, the experience of having a

personal response made the students have their own preference.

From the observation, none of the ten tutors provided individual responses in the

discussions. Most of the tutors offer remarks or conclusions at the end of the discussion. Two

tutors involve in the conversation of the most interesting student posts that are responded by

many students. Two tutors, however, never gave any comment or responded to the discussions at

all, not even a highlight at the end of the discussion. Although the courses that have been

observed are probably not related to the student comments, the students likely have the same

113
experience with the tutors on other courses they participate. As one student says, “Some tutors

need to be more actively involved in the discussion, not just post discussion question and then

disappear, at least they can give a summary or conclusion.” Another says, “Some tutors are not

responsive and I did not feel that I was facilitated; the tutor did not really do their job.”

From the ten courses observed, the students posted a comment in response to the

discussion question posted by the tutor. The student opinions or posts sometime draw attention

other students to comment in the posts. In this manner, the discussion actually happens among

the students. The interaction among the students, however, does not satisfy the students who

expect a response from the tutor. “I am not sure about the opinions from other students since they

are also students who are in the process of learning” remarked one student. Related to the lack of

tutor responsiveness in the discussion sessions, all the tutors admit that they do not give

individual response in the discussion session. Four out of six tutors say that they do not think

they have to respond to every post sent by the students. It is likely impossible to respond every

post since the number of students in every class is large. The tutors feel that not every post needs

to be responded to. They say that providing general comments or conclusions are enough to

highlight the essence of the discussion topic. Moreover, the tutors also reason that giving

personal responses to every post is impossible considering the number of the students in every

group and also the number of classes they have to manage. As indicated in the Table 6.2, each

tutor has to manage at least 4 classes in the semester 2012.2.

Two of the tutors, however, say that they wanted to respond to every student post

following what they used to do. These tutors remember that in the past they were able to answer

all student posts when the number of the students in the class was not as big as it is in the present

time. Tutor number 2 (T2) said that when she only had to manage one class with total students

114
around 60 students, she knew the all the students individually. She gave comments to each

student posting in the discussion. She also promptly answered any question from the students

related to the course content or other issues students identified. She said after the tutorial session

ended some students sent her emails if they had questions related to their study or simply to greet

her.

Now, when she has to manage five classes (and two classes of online tutorial in a

graduate program), it is impossible for her to know her students one by one. The volume of

studnets means that she can only skim the student posts in each class, except for two classes in

the graduate school because the tutor in this program needs to be more active and intent

interacting with the students in the online tutorial. In her five classes of bachelor degree tutorial,

she says, “I can only act as an administrator rather than as a facilitator.”

Her testimony refers to the fact that she cannot be the real tutor who is supposed to

facilitate and motivate her students in the online tutorial. She can only record the students who

submit posts in the discussion without giving proper feedback. She thinks that she cannot be a

good tutor anymore.

When the problem of interaction in discussion is clarified to a senior staff at the faculty

who is involved in the development of the online tutorial, she says that actually there no

obligation for the tutors to interact intensively with students in the discussion. This senior staff

member says that she actually prefers that tutors limit their involvement in the discussion merely

to facilitate and motivate students to be involved in the discussion but avoid sending too many

comments, except when the discussion need clarification, direction or further explanation. In her

opinion with the characteristic of Indonesian students who are used to passive learning, the

active involvement of tutor will prevent student activeness in discussion. She says that in the

115
middle of active discussion, if a tutor gives an opinion or suggestion that will thwart students

giving more opinions because the students will think that the tutor’s opinion is the right answer

for the discussion topic.

The confusion on the interaction between students and tutor in a discussion session in the

online tutorial represents the uncertainty of how the tutor should act in the discussion. Firstly,

this can be because the absence of basic policy of interaction between tutor and students in this

program. There are mixed feelings about the intensity of interaction between tutor and students.

On the one hand, this tutorial is on a voluntary basis and student can have a 30 percent

contribution to the final grade of the course. On the other hand, this program is actually cannot

reach all students. There is a feeling that it will be unfair for students who cannot participate in

the program because they have no access to the internet in the areas where they live. The

authority in the faculty says that this program needs adjustment to find out the best model for UT

student characteristics, student interest to the program, and the availability of resource support to

the program.

Secondly, the rapid growth of students interested in the program also significantly

influences the ability of tutors to interact more intensely with the students in the discussion. With

the tutor workload to manage many large classes, it is likely impossible to require tutors to be so

involved with their students in discussion. Thirdly, it is probably time for UT to decide what kind

of interaction they intend to provide to students considering the great demand of the program and

the unavailability of resources to support it.

Issues of Feedback and Grading

Tutor feedback. The other thing that students expect is feedback on the assignments they

submit and the online duties they perform. All the students interviewed expect that the tutor will

116
give feedback on their assignments. They anxiously want to know whether their assignment

answers the tutor’s question for the assignments. Yet the students have to hold their

disappointment with the inability of tutor giving feedback. In the interview, a couple of students

say that they are unsure of the points they receive for assignments or discussion posts and are not

even sure that the tutor received their submissions.

Some students apparently had the experience of having feedback on the assignments

previously. The students explain that in earlier semesters tutors in the online tutorial made

comments and suggestions on their assignments, so they fixed the assignments and returned them

to the tutor to get a better score. One student say, “… previously, some tutors gave feedback by

making correction on my assignments and gave them back to me for revision.” Another student

supports the student’s experience and claims that, “Two tutors gave me feedback on my

assignments last semester, provided comment and informed the score.” Recently, however, tutors

never give them any information about their assignments. This situation makes them worried that

they will get low scores on their assignments.

When the student complaints are brought to the tutors, they affirm that they are not able

giving feedback in the assignments. The huge number of total assignments they have to read is

too overwhelming. Not to mention that they have to supervise the discussions. With other

responsibilities as faculty members, they do not have enough time to make comments and

suggestions on each student assignment for revision. They say that they can only read the

assignments and score them.

Two tutors say that they used to made some comments and suggestions on the student

assignments and gave the assignments back to the students for revision as the students desired.

One of the tutors even mentions that she also gave suggestions about the relevant references in

117
the student assignments. She did this when she only dealt with one class tutorial in the past. The

students who recently complained about the feedback on assignments are perhaps the students

who are facilitated by this hard-working tutor.

It is actually understandable that currently all tutors have been swamped by many large

classes to manage. Hypothetically, if a tutor manages four classes with 100 students active in

each class, the tutor has to handle 100 assignments three times for four classes. Meaning that the

tutor has to read and give feedback for 1,200 short essay assignments in five weeks, with the

consideration that the first assignments are received by the tutor at the end of the third week of

the tutorial. It is likely impossible for the tutor to maintain good work with such huge work

burden. In the reality, many tutors actually have to manage more than four classes which mean

that the tutors have more workload than the hypothetical calculation.

It is likely that the Faculty and University have to redefine their strategy maintaining the

huge number of students and vast number of online tutorial classes with limited tutors. One way

to clear the student confusion is by developing good guidance which clearly describes the role of

the tutors in the online tutorial and the Faculty expectation to students in the every activity in the

tutorial. The clear guidance and explanation of each activity in the tutorial will provide

information of the real situation on the tutorial to the students. The students thus will not have

heightened expectations.

Grading transparency. Other concerns have to do with clarity and transparency in the

grading process. The online tutorial score and grade are the matter that the students anxiously

want to know. A contribution of 30% to the final course grade is one of the biggest motivations

for the students to join the tutorial. The student, who passively participates in the tutorial by

accessing the website, reading the initiation materials, questions, and responses from tutor and

118
students will have 20 percent score. If the student actively participates by posting comments,

responding to questions and comments from the tutor and other students, his/her score may

increase up to 50%. To get maximum 100% score the student has to complete all three

assignments in addition to active participation in the discussions.

Currently, most tutors are unable to provide information about students’ scores. As the

result, students wonder well they are performing. The students speculate whether their

participation on the online tutorial is actually useful in improving their final score, as one student

says: “I wonder if my final course grade includes the contribution from the online tutorial.” The

same question is also addressed by a student, “I never know how the tutor scores my

participation for discussion and assignments. The tutor does not allow me to know my score.”

Tutors admit that they are unable to provide feedback because of the heavy workload.

Some tutors put the first and the second assignments scores in the score box in the website, but

for the third assignment and the final scores they are unable to tell the students. When one tutor

finished scoring, it was already past the end of the tutorial. Most of the tutors argue that scoring

is time consuming work, especially when they have to do manually. There is software on the

website to record each score, but a tutor says that they have to calculate the final score manually.

The main problem is that the tutors have to do the scoring for many classes. One tutor says that,

“One course, takes at least three days to recap each student’s participations in the discussions and

then calculate with assignment scores into final score.”

In the given box of scoring on the website, there is only one box for the final participation

score for each student. It seems that the tutor has to track the participation of each student in all

discussion sessions to determine the final score of participation. Unfortunately, not all students

actually participate in all sessions of discussion and the tutor has to scrutinize for each student in

119
each session of discussion. Counting the participation for each student in each session of

discussion indeed takes some time. Later the tutor has to combine the participation score with

the assignment scores for the final score of the tutorial. As one tutor says, if one class takes three

days of scoring, for five classes she maintains she needs 15 days to finish all the jobs. In the

reality, she might spend much more than 15 days for the scoring job since she also needs the

time to handle her other responsibilities as faculty member. Because of this, she is not able to

provide the final score of the tutorial to her students directly.

Another tutor says that she used to provide the score to the students before she sent the

score recap to Examination Center. However, doing this was much more convenient and less

time consuming in the past. She even provided explanations when students questioned how she

calculated the final score for the program. Currently, however, she is afraid that she will have no

time for such explanation.

Agreeing with the tutor some students also mention that in the previous semester some

tutors provided the tutorial score to them, although the tutors mentioned were probably not the

same tutors. This expectation is expressed by a student, “In past semesters the tutors put the

scores on the website so we knew at least our final score in the tutorial.”

It is likely very important to the program to provide students their scores since the biggest

motivation for student participation in the program is having a good grade in the tutorial. A good

score from the program is expected to improve student final course score when his/her final

exam score is not good enough. Also, it may be very useful if the system can provide a tool

where the tutors need not calculate scores manually so that the tutors can save time on scoring

and have time to provide scores to the students.

120
Workload and Schedule Concerns

Both students and tutors in the online tutorial bear a heavy burden of occupation within

and beyond the UT program itself. Issues posed by this fact were frequently raised in the

interviews.

Student workload. The design of the online tutorial in Universitas Terbuka is generated

from the design of the face-to-face tutorial. In the face-to-face tutorial student participants have

to attend eight classroom meetings and complete three assignments in an eight-week tutorial

session. The maximum number of students in one face-to-face class is only 20. Similar to the

face-to-face tutorial, the online tutorial is also designed in the same model that students

participate in eight discussions and complete three assignments. In the early implementation of

the program, the number of the students in one group/class was not as many as recent years. In

one class the number of students was possibly less than 50 students. The communication

transaction therefore was not as crowded as in the recent group with 300 students. The tutor’s

attention to each student could be also more intense. Moreover, in the past the number of courses

that offered online tutorial was small. In one semester, a student perhaps only took two or three

tutorials from the total courses he/she registered. In recent years when all courses offer an online

tutorial, if a student takes eight courses as maximum courses in one semester and the student

joins in all online tutorials, it seems impossible for the student to complete all tasks in the

tutorials.

Compared to the face-to-face tutorial, the tutor in this tutorial only has to manage 20

students. In the classroom meetings the students have opportunity to interact with the tutor much

121
more intensively, not only related to the content of the tutorial but also any other matter related

to the students learning. Any confusion on the tutorial task and information can be clarified by

the tutor instantly. The tutor also has a considerable workload with the 20 students. More than

that, the final score of the face-to-face tutorial can contribute 50 percent toward the final course

grade. While for the online tutorial, with approximately 150 students active in one class

(assuming that 50% of maximum 300 students are active in the class), each student barely gets

enough attention from the tutor. Unclear instruction cannot be immediately resolved if the tutor

does not have enough time to answer every inquiry from students in the class. Not to mention if

the students miss time because they cannot access the website for several reasons. Combined

with the tutor’s heavy work load, the students in the online tutorial are unintentionally neglected.

The service may be is not as effective as meant to be.

A student from Palangkaraya mentioned in an interview that because she took six online

tutorials in the semester, in each week of assignments she had to finish six short essays in a

week. With internet connection issues, she said that she could not complete all the assignments

of six tutorials she was enrolled in. Hence, she could only complete the assignments of three

tutorials, and the other three she simply participated in the discussion session. With her inability

to complete the assignments in the three tutorials, she expected that her only participation in

discussion could still improve her score in the final course grade.

The case of the student from Palangkaraya illustrates that student motivation to complete

all tasks in the tutorial is high, but the many tutorials she joins made her unable to complete her

tasks entirely because of the time constraint. On the other hand, the Faculty cannot suggest to the

student to limit the number of tutorials a student participates in since the service is a free option

and the student thinks the tutorial is useful for her.

122
 Examining the field situation, although there is no specific complaint from students about

the task load, it seems that the increased number of online tutorials offered is not quite effective.

It is necessary for the University to redefine the students work load for some considerations:

1. The large number of students in one group that need attention from tutor.

2. The time frame on completing each assignment which students think is inadequate,

particularly with the constraint of internet connection problems.

3. The large amount of online tutorial tasks of the student taking highest number of

courses in one semester in the eight-week duration of tutorial.

4. The task of scoring all of the work for four or more tutorial classes in one semester.

5. The contribution of 30% to the online tutorial final course grade compared to 50%

contribution for the face-to-face tutorial.

The reduction or modification of student tasks probably will make the program more

effective for the students and the tutor work load may be reduced. The modification can be done

by:

1. Reducing student assignments from three to two.

2. Giving more time for student to complete each assignment.

3. Modifying the short essay assignment to multiple choice formats for simpler

correction job.

4. Some combination of the options.

Task timeline. Another criticism voiced by the students concerning the activities in the online

tutorial is the task timeline. Some interviewed students complained about the short duration of

the tasks in the tutorial, particularly the assignments. A student complains “We need more time

to complete each assignment, one week is not enough.” The difficulty of accomplishing the

123
assignment, furthermore, is not only related to the limited time but also because the bad internet

connection is often the reason that the students lost a lot of time to complete the assignments, as

another student argues, “One week for each session is too short. When we have problems with

the internet connection or accessing the website, we lose a lot of time particularly to complete

our assignments.”

This is a real problem which the Faculty should consider students completing all the tasks

in the tutorial. It is quite reasonable that the students need more time for their tasks particularly if

they have to deal with the issue of internet reliability that they have no power to control. Also,

internet service is uneven in Indonesia especially in remote areas. The tutors who mostly work at

the head office probably never realize that the problem of internet accessibility and reliability in

many places in Indonesia is a serious problem for students completing tasks in the tutorial.

When students report problems with their internet connection, some tutors are responsive enough

to allow extensions, but some other tutors remain firm with the timeline. As a result, some

students fail to complete their assignments. Perhaps these tutors do not realize the problems

faced by the students. A student shares his experience, “Some tutors are very strict with time,

when I have problem with my internet connection I lose a lot of time to complete the

assignments. When I finally finish the assignment, I cannot submit it because the box has been

closed.” It is like necessary for UT to modify the policy relates to time frame of assignment

submission, as a student suggests, “UT needs to fix the policy about the duration of completing

the assignments because one week is not enough.”

It is important for UT to consider the duration of each task in the tutorial by taking into

account the slow connection of the internet as the common problem of student, besides one week

124
is probably too short for them completing the task, especially if the students have to use outside

references for the task.

Tutor workload. In the second semester of 2012 (2012.2) there were 264 online tutorial

classes of bachelor degree courses and 176 classes of diploma courses. In that semester most of

the tutors, who maintained four classes or more, felt that they did not perform well because of

the overwhelming work load. In the first semester of 2013 (2013.1) a new policy from the

Government through the Department of Education and Culture will be applied to the policy of a

single tuition fee in the higher education institution in Indonesia. This policy aims to prevent the

practice of taking additional fee from students when the university offers additional learning

services. As consequence of this policy, the same tuition fee for the similar course must offer

similar service. Meaning the same online tutorial service must be offered by all courses. As a

result, all courses at the University are now offering an online tutorial. In 2013.1 the numbers of

classes of online tutorials for a bachelor’s degree in FISIP have increased to 319 and the numbers

of online tutorials of the diploma courses have reached 212 classes.

The sudden increases, however, does not match with constant number of tutors which

actually have been already weighed down in the online tutorial last semester. To solve the

overburdened tutor problem, the Faculty decided to hire tutors from other universities and other

relevant educational institutions, actually this strategy had been tried in the 2012.2 semester. The

University confines the number of the online tutorials to four classes for each tutor. The rest of

the classes will be managed by outsourced tutors. In this transition some problems may appear

about the outsourcing of tutor as predicted by some tutors and administrators:

125
1. The tutors are hired from conventional universities and they are not familiar with the

online teaching or tutoring format. Anticipating this problem, these tutors would be

trained to make them familiar to with the new tutoring format.

2. The tutors are not used to with the work regularity of distance education which

usually needs to be available at various times, even sometimes after office hours. This

would be different than the teaching culture in a conventional university which the

teaching hour is in exact time in class.

3. In anticipation with the unfamiliarity of the new tutors with the distance education

system, the internal UT tutors are assigned to supervise the new tutors. This means

that the UT tutors will have the additional task supervising their new partners.

4. The culture of the Indonesian bureaucracy might complicate the tutor outsourcing

project. In Indonesian universities, qualified and senior lecturers are a highly

respected profession. Supervising those senior qualified lecturers as tutors might be

not so easy for UT tutors who are usually not as old in age as the senior professors

from the conventional universities. As illustrated by an UT senior tutor in the

interview, in 2012.2 when the initial outsourcing began, there was an agreement

between FISIP as the hiring agent for the outside tutors and the tutors that all

information and reminders will be sent via short message text. However, when the

outsourced professor got a reminder from FISIP in his cellular phone, the senior

lecturer felt offended and treated in disrespect manner. As a result, the tutor from UT

had to take over the tutorial although the tutor already had his own online tutorials to

maintain.

126
 Various obstacles will likely have to be faced by administrators and tutors at UT dealing

with the tutor outsourcing project. It will take time to get everything organized as set up to make

use of the newly hired tutors. This means the students might not be served well during the

transition time before everything gets settled.

Program Results

The Overall Usefulness of the Online Tutorial

The essential topic related to online tutorial results discussed during the interviews was

the overall usefulness of the program to students and stakeholders. Although there are many

complaints from student on the implementation of the online tutorial, all the students in the

interview thinks that the tutorial is useful for their learning. Although the students say they never

know their grade in the tutorials, the student believes that her participation is improving her

course grades. This student says,

Although I missed the feedback from the tutors, when I actively participated in
discussions and completed all the assignments, my final grades were good…. but
when I was busy and I rarely posted comments in the discussions and I did not
complete the assignments, then my grades were disappointing. Therefore I assume
that the tutors did their job and the tutorial is actually useful for my study.

The other dimensions of usefulness mentioned by the students can be listed as follows:

1. The online tutorial helps students understand the content of the printed materials. As

two students testify:

I live in a small town surrounded by forest in Central Borneo, although the

printed material is in Bahasa Indonesia, it is not easy for me to study alone with
my limited understanding of academic language in the printed material.
Discussions in the online tutorial help me to clarify the concepts in the material.

Doing the assignments forces me to search and read more reference and make me
have better understanding the essence of the printed material.

127
2. The online tutorial that requires students to read the introductory and printed material

every week which is very helpful for final exam preparation. One student says, “The

online tutorial pushed me to read the printed material earlier, long before the final

exam.” Meaning that the student continually reads the printed material little by little

along with the progress of online tutorial session in agreement with a student claim,

“The online tutorial helped me prepare the final exam; it pushed me to read the

printed material little by little.” Another student who usually reads the printed

material at once in the near time of final examination asserts that participating on the

tutorial motivates him to read the material earlier, “Without the online tutorial, I am

not motivated to read the material early. If there is no online tutorial I would only

read the printed material a week before the final exam time.”

3. The contribution of 30% to the final course grade is a big motivation for students to

continuously participate to the online tutorial.

4. The online tutorial provides various methods of study to the students. They do not

have to always read the material alone. One student says, “Without the online tutorial

I feel that I study alone and I have nobody to talk to when I have problems with my

study.”

5. Indonesian students who are used to with the conventional education system, think

that distance education is not real education. Participating in the online tutorial

makes them feel that they are receiving a real education. “The online tutorial gives

me feeling that I am in a real school” remarked one student.

6. The common perceptions of students who study in the distance education system are

the feelings of isolated and lonely. The online tutorial which provides brief student

128
 identification allows the students to know the email, contact number, and where the

student lives. Some students who live in the same area can contact each other or

even study together. The online tutorial therefore reduces the feelings of isolation as

students say, “The online tutorial gives me a chance to know other students in the

group. We sometime have meeting in person to study together, share experience, and

resolve study problems.”

7. Lastly, although the students have some critiques and suggestions over the

implementation of the online tutorial in general they think that the online tutorial is

helpful for their learning. Some students are planning to take a face-to-face tutorial

for some courses, but they say that they will also take online tutorials. They might

not be able to complete all tasks in the tutorial but they can still read the materials

and posts from the tutor and other students. As a student says “I am taking eight

courses in 2013.1 and taking four of face-to-face tutorials, but I will take all online

tutorial too.”

129
 CHAPTER SEVEN

CONCLUSIONS AND RECOMMENDATIONS

Introduction

When Universitas Terbuka was first launched, it could never have been predicted that

information and communication technology would develop as widely as it has and the internet

has become the major distance communication media for the educational system. In its early

establishment, UT founders believed that face-to-face tutorials and student study groups were

destined to be the major learning supports for UT students, despite the existence of

correspondence strategies and nascent electronic tutorials. Until recently, the face-to-face tutorial

was the most favored learning support for students. Due to the individual methodology of the

tutorial by correspondence and the one-way communication of initial electronic learning support

services, these support alternatives have, however, become unpopular among the students.

Although the face-to-face tutorial continues to be the preferred learning support, this

service can only reach a limited number of students who can afford the fees and have the time

and ability to travel to the tutorial location. The rest of the students – in fact, the large majority --

can only stick to the printed material and the electronic learning supports. The emergence of the

internet and the development of communication technology infrastructure in Indonesia in the end

of the 1990s had given hope for UT to develop an effective and efficient student learning support

that could reach broader students. UT was anxious and hopeful at the same time at the

introduction of the service. The anxiety could be because UT spiritedly offered the service but

the student response was not very enthusiastic. UT was also hopeful because it is believed that

the service will eventually better known and used by more students aligned with the expansion of

ICT infrastructure development.

130
 Until 2007 student interest in the program was still moderate, although there was a

constant increase in the number of the participants. In accordance with the vast global

development of ICT the number of internet users in Indonesia also dramatically increased as well

as the number of students who joined the online tutorial. With the emergence of the flash modem

and the smart phone, the increasing number of students who participate has exceeded

expectations. As that number grows, it is likely that the model developed years ago for online

tutoring is no longer suitable or adapted to the current situation. Problems with finding tutors

have now been addressed by outsourcing tutorial jobs, but that is scarcely an overall solution to

the need for updating the program model and strategy.

The immediately preceding chapters of this dissertation have presented a variety of data

and analyses concerning the fidelity of the implementation and the effectiveness of the current

UT online tutorial model. This last chapter is devoted to drawing a first set of conclusions and

formulating a few recommendations for updating and improving the UT online tutorial model.

Conclusions

This study is scarcely a definitive assessment of the current state and future possibilities

of the UT online tutorial. Conclusions therefore must remain tentative and suggestive. The most

important of those evident to the author on the basis of the data examined are the following:

1. The context for development and implementation of the bachelor degree of online tutorial

is a difficult and demanding one, given the enormously increased demand for the

program over recent years and the need therefore to gear up rapidly in an environment

where not all necessary resources are in place.

2. Though a number of manuals for online tutorial implementation and guidelines for

training or tutors have been established, it remains a little difficult to describe with

131
 precision the intervention model of the program. There are areas where specific methods

are unclear and others where existing procedures seem outdated and necessary guidance

appears to be lacking.

3. The program is nonetheless implemented with a great deal of consistency among the

courses following the stated methods with little variation in the practice of different tutors

either in the way in which they carry out those activities -- or in their omission of

approaches not specifically required and explained in their training. This consistency,

however, does not necessarily imply that the practices are of good quality or effective.

4. Students complain about certain deficiencies in program administration and in the

conduct of tutorial sessions, among them the lack of transparency in grading procedures,

the general absence of feedback on assignments submitted and the lack of much online

interaction with tutors.

5. To judge by tutor and administrator comments, many of these problems can be attributed

to the fact that instructional and administrative staffs are swamped by the greatly

increased numbers of students. Tutors often must supervise three or four tutorials with up

to 300 students at the same time, which just about precludes any interaction or individual

attention.

6. Despite these concerns and shortcomings, test results show that in the majority of courses

(six of the ten sampled) students who have completed the online tutorial score

significantly better on the final exam than those who have not, though the data do not

enable us to control these results for SES or rural-urban location.

Overall, the online tutorial program appears to be performing a real service but is in need

of better specification, some modification of methods and closer quality control.

132
 Recommendations

The conclusions just enumerated issue a certain number of final recommendations:

Redefining Style of Online Interaction

The online tutorial has been offered for about 15 years since its initiation with a pilot

project. The current model basically was designed about 10 years ago when the environment was

quite different. Since the old model does not seem appropriate anymore to the present condition,

it is time for UT to redefine the model to adjust with the current capability. The greatest

deficiencies of the model are the inability for the tutor to give intensive interaction with the

students, to provide proper feedback of the assignments, and to inform the activities and final

scores of the tutorial. Outsourcing tutors may be one way to resolve the problem, but this will not

assure that the hired tutors will commit well in the program since they are not used to with the

work habit and culture in distance education system. It is likely UT should find an alternative

method of delivery the service.

The popularity of open educational resources programs such as Open Courseware

program offered by MIT or Massive Open Online Courses (MOOC) program can probably

provide insight to UT about how to modify and adjust the recent the online tutorial to becoming

UT open courseware with specific specifications and tailored to UT’s style for student learning

service.

The school’s Open Courseware Web of MIT provides access to syllabi, lecture notes,

assignments, exams, problem and solution sets, tools and tutorials, and a growing library of

video lectures for most of MIT’s courses. While MOOCs are online courses that are free to

anyone. MOOCs lectures “are typically “canned”, quizzes and testing are automated, and student

participation is voluntary” (http://net.educause.edu/ir/library/pdf/PUB4005.pdf). MOOCs reach

133
enormous number of students by reducing instructor interaction with student individually;

students independently organized themselves in study and discussion. Another example is Open

University from the United Kingdom that also has MOOC called OpenLearn. The OpenLearn

gives free access to learning materials from The Open University. The participant accesses and

learns all materials as the regular The Open University students for free but there is no tutor to

contact or interact with. If the participant wants to take an exam and gets qualification, then the

participant has to pay the fee.

UT can possibly modify the recent online tutorial by providing various good materials

related to the course, continue to provide discussion facilities and assignments but UT has to

redefine the intensity of interaction between student and tutor and whether the tutor will give

feedback on the assignment. All the activities have to have clear guidance and instruction. The

adjustment program, of course, has to be indicated clearly to students, so their expectations will

be tempered on all activities to the new online tutorial. UT can create its own specifications for

this learning service to give as much opportunity as possible to the students without giving more

burdens to the tutor or the University when the number of participants reduces or increases.

Creating Manual of Implementation

The most essential item that is missing from the online tutorial program is a manual or

handbook of implementation. The manual is urgently needed particularly for the tutors who have

no basic skill or knowledge about teaching online or in e-learning situations. The basic training

that they attended was training for a face-to-face tutorial which is significantly different from the

online tutorial. The handbook can be a general online teaching guide and provide specific

information for the online tutorial.

134
 The general online teaching handbook can give an overview to tutors about how to teach

in an online environment. The handbook can include an overview of teaching in online learning

environments, teaching online courses, the challenges of teaching and learning online, assessing

student learning online, resources for online learning, and other necessary issues in online

teaching and learning. The handbook will guide tutors through the decisions that they need to

make when tutors are involved in online teaching.

The specific handbook should cover step by step the implementation of the online tutorial

program. The handbook for tutors may include tutoring preparation, the process of online

tutoring, and reporting the results of the online tutorial. The guideline for online tutoring should

cover the details of tutoring methods and strategies on opening the class, preparing and

facilitating discussion, assessing student learning, and closing the tutorial class. The guide book

should include the detailed methods and steps of online tutorial teaching with specifications

about the design program to avoid confusion among the tutors. Of course, there should be room

for tutors to adjust the method and strategies of tutoring with the nature of the course and

teaching environments.

It has to be realized that although many students are used to UT’s distance learning

method and style, the students most likely have no idea about learning in online environments.

The handbook for student learning in the online tutorial is necessary as well. The information in

the handbook may include the orientation for student learning in an online atmosphere. Students

new to online learning in the tutorial may initially find this kind of learning disorienting. The

orientation can provide information on the differences between learning online and learning in

classroom. Preparing students for the amount of time needed to take the online tutorial, the

material they should have, the necessity of reading and writing in the online tutorial, and

135
interacting with other students and the tutor in discussion sessions. The other important element

in the guideline is the clarity of expectations for students to perform well in the tutorial so they

will gain a good grade that can significantly contribute to their final course grade.

Upgrading and Intensifying Tutor Training for the Online Environment

As mentioned above, the basic educational experience of most online tutors, who are

academic staff, is with traditional classroom education. The basic tutor training requirement to be

eligible as tutor is also training for the face-to-face tutorial. Even for senior tutors, skill is gained

by experience after years being an online tutor. It is therefore very important for the University

to upgrade the tutors’ skill with pedagogical online teaching training.

Although there are many versions of instructor roles in online teaching, the role of

instructor teaching in the online environment is different from the one in the traditional

classroom. In Berger’s (2000) conception, for example, the roles of instructor in online teaching

include four dimensions: pedagogical, social, managerial, and technological. Naturally the

instructor roles in the online teaching environment are different from those in the classroom

setting since the online setting involves media and communication technology. In the online

environment they have to be able to make participants comfortable with the system and software

program used in the online tutorial. The bottom line, online teaching training is compulsory to

the tutors.

Reviewing and Lightening Student Workloads

The other concern that urgently needs attention in the program is the student assignment

load. The design of the online tutorial was generated from the face-to-face tutorial. The basic

model is eight classroom meetings and three assignments in the face-to-face tutorial which are

adapted to eight week discussion sessions and three short essay assignments. The difference

136
between the two kinds of tutorials is that the score of the face-to-face tutorial may contribute

50% to the course grade, while the score of the online tutorial is only worth 30% for course

grade contribution.

The model that was designed more than 10 years ago remains the same for the face-to-

face tutorial. One class of a face-to-face tutorial consists of a maximum of 20 students with one

instructor. The students in the face-to-face tutorial have the opportunity to interact and consult

with their tutor more intensively when they have problems in learning including how they should

complete the assignments. For the face-to-face tutor the work load to facilitate and give feedback

to students also the same at all time. While for online tutorial students, the assignment load also

stay the same but their opportunity to interact and consult with their tutor is very limited,

particularly with the current condition when the tutors have so many students to manage in the

class.

Moreover, while offering the online tutorial of all courses; most students interested in the

program tend to participate in all tutorials of courses they take. Which means the students will

have to complete a bunch of assignments in the week of assignment session. This pattern of

student assignment in the online tutorial is likely to be revised by giving a reasonable workload.

The modification can be reducing the number of assignments or adjusting the assignments to

those that are less time consuming for the students to finish. The modification is needed to

respond student suggestion and also to reduce tutor workload in scoring. Otherwise, the

assignment model will not be effective when the students are unable to complete them.

Better Specifying the Nature of Student Syllabus and Guideline

The syllabus for students in the online tutorial is in the format of Tutorial Activity Design

(Rancangan Aktivitas Tutorial/RAT) and Tutorial Activity Unit (Satuan Aktivitas Tutorial/SAT).

137
Most tutors upload Tutorial Activity Design in the website so the students can download or read

it. This format is originally designed for the instructor for tutoring guidance. Although in the

design plan written information on the objectives of the tutorial, topics, and subtopics; this is a

technical format that the students do not understand. It is doubtful that the students read or

download the syllabus. It is suggested therefore to change the format of the syllabus into more

readable format for the students.

In each week of the tutorial session, it will be more helpful for students if the tutor not

only uploads the introductory material and discussion question but also describes in more detail

the weekly objective, the topic, and the competence that will be achieve in the week along with

the initiation material. Thus the students know clearly the direction of the discussion and the

competency they should gain in the session activities.

Other than that, it is very important for the tutor to provide rules and strategies in the

discussion that includes how may posts the each student should send, the duration of the

discussion, and the possibly score the student would gain if they complete the tasks in the

discussion.

Such rules and strategies are also necessary to be given in completing assignments. The

tutor should provide information, the length of the essay, the duration, the format of the writing,

and the possibly score the student will gain if the fully complete the assignment.

Further Investigation

Despite the deficiencies of some areas indicated by the students, there are also

optimistic results for the online tutorial program. Analysis shows that the program is actually

useful. The statistical overall test results also demonstrate the effect of the program in improving

student scores on the final exam. The individual course t-test results, however, demonstrate that

138
only half of the sample tutorials have significant difference and the other half have no significant

difference between the means of the two kinds of final examination scores. It is very important to

conduct further investigation on the effect of participating in the online tutorial to the course

final exam scores to find out other shortfalls of the program for further adjustment and

modification

Hopes

Hopefully careful attention to some of the data gathered for this dissertation and careful

consideration (though not necessarily wholesale adoption) of the recommendations offered above

will contribute to achieve the objective that we all share: improving the quality of online the

tutorial service delivery and better meeting the needs of UT students.

139
 APPENDIX A

INTERVIEW PROTOCOL WITH STUDENTS

1. Please introduce yourself and tell me how long (how many semesters) have you been

joining the online tutorial?

2. Did you have experience of using Internet before you joined the online tutorial?

3. How many online tutorial you participate this semester?

4. Is there any guide book or leaflet that shows you what to do in the tutorial?

5. When and how do you usually participate in the tutorial(s)?

6. Did you get sufficient feedback from the tutor in the tutorial?

7. What is the strength of this tutorial?

8. Is there any weakness?

9. Are you satisfied with the tutor performance?

10. Do you have any suggestion for the tutor in maintaining the tutorial?

11. Where do you usually access the online tutorial site from?

12. Have you ever have problem with the connection or accessing the tutorial site?

13. If so, where did you get help to solve the problem?

14. What is the challenge(s) of participating in the online tutorial?

15. Do you think the online tutorial help you in learning?

16. Do you think online tutorial help you preparing the final exam?

17. Is there any suggestion or recommendation to improve the online tutorial?

140
 APPENDIX B

INTERVIEW PROTOCOL WITH TUTORS

1. Please introduce yourself and tell me how long have you been a tutor of online tutorial?

2. Did you receive any training to be a tutor of online tutorial?

3. If so, what kind of training?

4. Is there any manual instruction or guide book

5. How many course of online tutorial you have to maintain every semester?

6. How many students in one group in average?

7. Did you develop a plan or material before the tutorial session?

8. Please describe your strategy tutoring the group: introduction, setting the tutorial and

discussion rules, presenting the plan, posting the materials, giving feedback?

9. How do you encourage the students to be active in the tutorial?

10. Do you give feedback in the discussion sessions or assignments?

11. How often do you access the tutorial site?

12. What is the most difficult challenge(s) maintaining the online tutorial?

13. How do you address this challenge?

14. What is the strength(s) this online tutorial?

15. I s there any weakness(s)?

16. Do you have other responsibilities other than tutoring?

17. Is there any suggestion or recommendation for the tutorial improvement?

141
 APPENDIX C

INTERVIEW PROTOCOL WITH ADMINISTRATORS

1. Please introduce yourself and how long you have been a technical support for online

tutorial?

2. What is your main responsibility as a technical support staff?

3. How do you maintain the online tutorial network in daily operation?

4. What is the challenge(s) of maintaining technical system of the online tutorial?

5. How have you addressed this challenge(s)?

6. Has any problem(s) or difficulty(s) ever reported by student and tutor?

7. Is there anything you would like to add about maintaining the online tutorial system?

142
 APPENDIX D

HUMAN SUBJECT COMMITTEE APPROVAL LETTER

The Florida State University

Office of the Vice President For Research
Human Subjects Committee
Tallahassee, Florida 32306-2742
(850) 644-8673 • FAX (850) 644-4392
APPROVAL MEMORANDUM

Date: 2/11/2013

To: Ida Zubaidah

Address: 160 Crenshaw Dr. #1, Tallahassee, FL 32310

Dept.: EDUCATIONAL LEADERSHIP

From: Thomas L. Jacobson, Chair

Re: Use of Human Subjects in Research

Evaluating the Implementation of the Online Tutorial for the Universitas Terbuka Distance
Learning Bachelor Degree Program in Indonesia

The application that you submitted to this office in regard to the use of human subjects in the
proposal referenced above have been reviewed by the Secretary, the Chair, and one member of
the Human Subjects Committee. Your project is determined to be Expedited per per 45 CFR §
46.110(7) and has been approved by an expedited review process.

The Human Subjects Committee has not evaluated your proposal for scientific merit, except to
weigh the risk to the human participants and the aspects of the proposal related to potential risk
and benefit. This approval does not replace any departmental or other approvals, which may be
required.

143
If you submitted a proposed consent form with your application, the approved stamped consent
form is attached to this approval notice. Only the stamped version of the consent form may be
used in recruiting research subjects.

If the project has not been completed by 2/10/2014 you must request a renewal of approval for
continuation of the project. As a courtesy, a renewal notice will be sent to you prior to your
expiration date; however, it is your responsibility as the Principal Investigator to timely request
renewal of your approval from the Committee.

You are advised that any change in protocol for this project must be reviewed and approved by
the Committee prior to implementation of the proposed change in the protocol. A protocol
change/amendment form is required to be submitted for approval by the Committee. In addition,
federal regulations require that the Principal Investigator promptly report, in writing any
unanticipated problems or adverse events involving risks to research subjects or others.

By copy of this memorandum, the Chair of your department and/or your major professor is
reminded that he/she is responsible for being informed concerning research projects involving
human subjects in the department, and should review protocols as often as needed to insure that
the project is being conducted in compliance with our institution and with DHHS regulations.

This institution has an Assurance on file with the Office for Human Research Protection. The
Assurance Number is FWA00000168/IRB number IRB00000446.

Cc: Peter Easton, Advisor

HSC No. 2013.9769

144
 APPENDIX E

CONSENT SCRIPT FOR STUDENT INTERVIEW

Hello, my name is Ida Zubaidah, a doctorate student in the College of Education at Florida State

University in the United States. I am conducting a research study to evaluate the process of

implementation of the online tutorial program for distance learning bachelor degree of Open

University of Indonesia.

I am asking you to take a part in this research study because I am trying to have

information about your opinions on the usefulness and effectiveness of the method and practice

of the online tutorial in helping your study; and your recommendations for the program

improvement.

If you choose to participate in this study, your participation will involve a 60-minute

taped phone interview which I will ask you questions in Bahasa Indonesia. All the information I

receivefrom you, including your name and any other identifying information will be strictly

confidential. You will not be identified in the tape or transcriptions of the interview. After the

tape has been transcribed, it will be erased.

There is no risk to participate in this study. Your participation will not bring any negative

effect to your study or any other risks. You will not benefit directly from participating in this

research study. However, the results of the study will contribute to the improvement of the online

tutorial program.

Your participation in this study is voluntary. You can decline to participate, and you can

stop your participation at any time without any negative consequences to you.

All research on human volunteers is reviewed by a committee that works to protect your rights

and welfare. If you have questions or concerns about your rights as a study participant, you can

145
contact the Human Subjects Committee, through the Vice President for the Office of Research of

Florida State University at (850) 644-8633.

Would you be willing to participate in this study? Would you like to do the interview

now or at a later time?

By you answering the interview questions I will ask, this means you consent to

participate in this research study.

If you have any questions concerning this research study, please feel free to contact me at

(850)339-0819 or by email at iz08c@my.fsu.edu.

146
 APPENDIX F

CONSENT FOR TUTOR INTERVIEW

Dear ___________,

I am a doctorate student in the College of Education at Florida State University in the

United States. I am conducting a research study to evaluate the process of implementation and
the result quality of the online tutorial program for distance learning bachelor degree of Open
University of Indonesia. Specifically, I would like to understand (1) how the Open University of
Indonesia prepares tutors in the online tutorial program; (2) the effectiveness of the present
online tutorial method and practice from tutors’ point of view; and (3) the recommendations that
are possibly given by tutors for the online tutorial program improvement.

If you choose to participate in this study, your participation will involve a 60-minute
taped interview which I will ask you questions in Bahasa Indonesia. You will not be identified in
the tape or transcriptions of the interview.

After the tape has been transcribed, it will be erased. Additionally, your participation is
voluntary and you have the right to withdraw your consent and discontinue participation at any
time. Your individual privacy will be maintained in all publications resulting from this study.

The risks associated with this study are minimal. I cannot guarantee that you will receive
any benefits from this study. However, the results of the study will contribute to the
improvement of the online tutorial program which will be more effective helping students in
their learning. If you have questions about your rights as a study participant, or if you feel you
have been placed at risk, you can contact the Chair of the Human Subjects Committee,
Institutional Review Board, through the Vice President for the Office of Research of Florida
State University at (850) 644-8633.

If you have any questions concerning this research study, please call me at (850) 339-
0819 or by email at iz08c@my.fsu.edu.

Sincerely,

Ida Zubaidah

*******

147
I give my consent to participate in this study. I also agree to be audiotaped, with the
understanding that tapes will be erased after the interview is transcribed.

________________________________ (signature) __________________________ (date)

148
 APPENDIX G

CONSENT FOR ADMINISTRATOR INTERVIEW

Dear ___________,

I am a doctorate student in the College of Education at Florida State University in the

United States. I am conducting a research study to evaluate the fidelity and nature of

implementation and the result quality of the online tutorial program for distance learning

bachelor degree of Open University of Indonesia. Specifically, I would like to understand (1)

how the Open University of Indonesia administratively and technically maintains the online

tutorial program; (2) the effectiveness of the present online tutorial method and practice from

administrators’ point of view; and (3) the recommendations that are possibly given by

administrators for the program improvement.

If you choose to participate in this study, your participation will involve a 60-minute

taped interview which I will ask you questions in Bahasa Indonesia. You will not be identified in

the tape or transcriptions of the interview. After the tape has been transcribed, it will be erased.

Additionally, your participation is voluntary and you have the right to withdraw your consent

and discontinue participation at any time. Your individual privacy will be maintained in all

publications resulting from this study.

The risks associated with this study are minimal. I cannot guarantee that you will receive

any benefits from this study. However, the results of the study will contribute to the

improvement of the online tutorial program which will be more effective helping students in

their learning.

If you have questions about your rights as a study participant, or if you feel you have

been placed at risk, you can contact the Chair of the Human Subjects Committee, Institutional

149
Review Board, through the Vice President for the Office of Research of Florida State University

at (850) 644- 8633. If you have any questions concerning this research study, please call me at

(850) 339-0819 or by email at iz08c@my.fsu.edu.

Sincerely,

Ida Zubaidah

*******

I give my consent to participate in this study. I also agree to be audiotaped, with the

understanding that tapes will be erased after the interview is transcribed.

________________________________ (signature)__________________________ (date)

150
 REFERENCES

Aji, S. S. (2007). The involvement of faculty members and distance students in online tutorial
program: Universitas Terbuka experience. Jurnal Pendidikan Terbuka dan Jarak jauh,
8(2), 114-120.

Ayuni, D and Helmiatin. (2007). Tutor activities in online tutorial for management courses at
Universitas Terbuka, Indonesia. Retrieved from asiapacific-
odl2.oum.edu.my/C24/F545.doc

Belawati, T. (1995). Increasing persistence in Indonesian post-secondary distance education.

Retrieved from http://www.pustaka.ut.ac.id/pdftesis/81200.pdf

Belawati, T. (2005). The impact of online tutorials on course completion rates and student
achievement. Learning, Media and Technology, 30(1), 15-25.

Belawati, T., Anggoro, M. T., Hardhono, A. P., & Darmayanti, T. (2002). Electronic Tutorials:
Indonesian experience. International Review of Research in Open and Distance
Learning, 3(1), 1-11.

Berge, Z. L. (2001). New roles for learners and teachers in online education. Retrieved from
http://www.globaled.com/articles/BergeZane2000.pdf

Bernard, R.M., Abrami, P.C., Borokhovski, E., and Tamim, R. (2009). A meta-analysis of three
interaction treatments in distance education. Review of Educational Research, 3(74),
1234-1289.

Beson, R., & Samarawickrema, G. (2009). Addressing the context of e-learning: using
transactional distance theory to inform design. Distance Education, 30(1), 5-21.

Caladine, R. (1993). Overseas experience in non-traditional mode of delivery in higher

education using state-of-the-art technologies: A literature review. Department of
Employment, Education and Training, Occasional Paper Series.

Chalmers, D., & Volet, S. (1997). Common misconceptions about students from South-east Asia
studying in Australia. Higher education Research & Development, 16(1), 87-98.

Dhanarajan, G. (2001). Distance education: promise, performance, and potential. Open Learning,
16(1), 61-68.

Dardjowidjojo, S. (2001). Cultural constraint in implementation of learner autonomy: The case

of Indonesia. Journal of Southeast Asia education, 20 (2), 309-322

Dunbar, R. (1991). Adapting distance education for Indonesians: Problems with learners
heteronomy and a strong oral tradition. Distance Education, 12(2), 79-88.

151
Educause. (2012). What campus leaders need to know about MOOCs. Retrieved from
http://net.educause.edu/ir/library/pdf/PUB4005.pdf

eMarketer. (2013). In Indonesia, a new digital class emerges. Retrieved from

http://www.emarketer.com/Article/Indonesia-New-Digital-Class-Emerges/1009723

Farkas, M., Simonsen, B., Migdole, S., Donovan, M., Clemens, K., & Cicchese, V. (2012).
Schoolwide Positive Behavior Support in an Alternative School Setting: An Evaluation
of Fidelity, Outcomes, and Social Validity of Tier 1 Implementation. Journal Of
Emotional & Behavioral Disorders, 20(4), 275-288.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative
approaches and practical guidelines. USA: Pearson Education.

Fulford, C. P., & Zhang, S. (1993). Perception of interaction: The critical predictor in distance
education. The American Journal of Distance Education, 7(3), 8-20.

Garrison, R. (2000). Theoretical challenges of distance education in the 21st century: A shift
from structural to transactional issues. The International Review and Research in Open
Distance Learning, 1(1), 1-17.

Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online

learning: Interaction is not enough. The American Journal of Distance Education 19(3),
133-148.

Gorsky, P. & Caspi, A. (2005). A critical analysis of transactional distance theory. Quarterly
Review of Distance Education, 6(1), 1-11.

Gravetter, F. J., & Wallnau, L. B. (2007). Statistic for the behavioral science. USA: Thomson
Wadsworth.

Guthrie, G. (2012). The failure of progressive classroom reform: Lessons from the Curriculum
Reform Implementation Project in Papua New Guinea. Australian Journal Of Education
(ACER Press), 56(3), 241-256.

Harvey, C., Killaspy, H., Martino, S., & Johnson, S. (2012). Implementation of Assertive
Community Treatment in Australia: Model Fidelity, Patient Characteristics and Staff
Experiences. Community Mental Health Journal, 48(5), 652-661.

Hillman, D. C. A., Willis, D. J., & Gunawardena, C. N. (1994). Learner-interface Interaction in

distance education: An extension of contemporary models and strategies of practitioners.
The American Journal of Distance Education, 8(2), 30-42.

Ika, L. (2012). Project Management for Development in Africa: Why Projects Are Failing and
What Can Be Done About It. Project Management Journal, 43(4), 27-41.

152
Islam, S. 2010. Kesiapan belajar mandiri mahasiswa dan calon potensial mahasiswa pada
pendidikan jarak jauh di Indonesia [Independent learning readiness of students and
potential students of distance education in Indonesia]. Jurnal Pendidikan Terbuka dan
Jarak Jauh [Journal of Open and Distance Education], 4(2).

Johnson, B., & Christensen, L. (2008). Educational research: Quantitative, qualitative, and
mixed approaches. Los Angeles: Sage Publications.

Jung, I. (2001). Building a theoretical framework of Web-based instruction in the context of

distance education. British Journal of Educational Technology, 32(5), 525–534.
Keegan, D. (1996). The foundation of distance education (3rd ed.). London: Croom Helm.

Mahle, M. (2007). Interactivity in distance learning. Distance Learning, 4(1), 47-51.

Maria, M., Zuhairi, A., and Riana, K. E. (2009). Students’ participation in online learning in
business courses at Universitas Terbuka, Indonesia. Retrieved from
http://www.ou.nl/Docs/Campagnes/ICDE2009/Papers/Final_paper_054mayamaria.pdf

Merriam, S. B. (1998). Qualitative research and case study applications in education. San
Francisco:Jossey-Bass.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. California: Sage
Publications

Moore, M. G., & Kearsley, G. (1996). Distance education: A system view. USA: Wadsworth.

Moore, M. G. (1973). Towards a theory of independent learning and teaching. Journal of Higher
Education, 44(9), 661-679.

Moore, M.G. (1989). Three types of interaction. The American Journal of Distance
Education, 3(2), 1-6.

Moore, M. G. (1993). Theory of transactional distance. In D. Keegan (ed.) Theoretical principles

of distance education (pp. 22-38). London: Routledge.

Nguyen, C. T. (2011). Challenges of learning English in Australia toward students coming from
selected Southeast Asia countries: Vietnam, Thailand, and Indonesia. International
Education Studies, 4(1), 13-20.

Rajan, S., & Basch, C. (2012). Fidelity of After-School Program Implementation Targeting
Adolescent Youth: Identifying Successful Curricular and Programmatic
Characteristics. Journal Of School Health, 82(4), 159-165.

Rumble, G. (2000). Student support in distance education in the 21st century: Learning from
service management. Distance Learning, 21(2), 216-235.

153
Rye, S. A., & Zubaidah, I. (2008). Distance education and the complexity accessing the Internet.
Open Learning, 23(2), 95-102.

Saba, F. & Shearer, R.L. (1994). Verifying the key theoretical concepts in a dynamic model of
distance education. The American Journal of Distance Education, 8(1), 36-59.

Scollon, R., & Scollon, S. W. (1995). Intercultural communication. Cambridge, USA:Blackwell.

Setijadi (1988) Indonesia: Universitas Terbuka, Prospect, 18(2), 189–197.

Sewart, D. (1993). Student support systems in distance education. Open Learning 8(3), 3-12.

Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluations, theory, models, and applications.
San Francisco: Jossey-Bass.

Starr-Glass, D. (2012). Learner perceptions of distance in an online course: Revisiting moore's

theory of transactional distance. Retrieved from http://www.umuc.edu/ctl/upload/Starr-
Glass_paper.pdf

Universitas Terbuka. (2002). Pedoman penyelenggaraan tutorial online edisi I. Jakarta:

Universitas Terbuka.

Universitas Terbuka. (2009). Universitas Terbuka 25 years making higher education open for all
Indonesians. Jakarta: Universitas Terbuka.

USAID. (1986). Indonesia: Education and human resources sector review. Vol.3. Florida: IEES.

The Jakarta Globe. (2012). Indonesia expects nationwide internet coverage by 2013. Retrieved
from http://www.thejakartaglobe.com/archive/indonesia-expects-nationwide-internet-
coverage-by-2013/510557/

Vrasidas, C. & McIsaac, M. S. (1999). Factors influencing interaction in an online course. The
American Journal of Distance Education, 13(3), 22-36.

Wong, J.K. (2004). Are learning styles of Asian international students culturally or contextual
based? International Educational Journal, 4(4), 154-166.

Yang, J. F. (2005). The challenges of Asian distance educational development. AAOU Journal,
1(1), 73-82.

Zuhairi, A., Adnan, I., & Thaib, D. (2007). Provision of student learning support services in a
large-scale distance education system at Universitas Terbuka, Indonesia. Turkish Online
Journal of Distance Education, 8(4), 44-64.

154
 BIOGRAPHICAL SKETCH

Ida Zubaidah

Born in Probolinggo (Indonesia), August 03, 1962.

Education

Ed.D. in Sociocultural and International Development Education Studies, Florida State

University, Tallahassee, Florida. 2013

M.A. in Sociology, Flinders University, South Australia, 1998.

B.Sc. in Agricultural Extension, Bogor Agricultural University (IPB), Bogor, West Java,
Indonesia, 1986

Professional Experience

Head Center for Community Services, Universitas Terbuka, 2005-2008.

Assistant of Vice Rector for Partnership and Institutional Building, Universitas Terbuka, 2003-
2005.

Head of Department of Sociology, Faculty of Social and Political Sciences, Universitas Terbuka,
2002-2003.

Secretary Department of Sociology, Faculty of Social and Political Sciences, Universitas

Terbuka, 1999-2001.

Academic Staff of Faculty of Mathematics and Natural Science, Universitas Terbuka (The
Indonesia Open University), 1988-present.

Publication

Teacher Training and transition in rural Indonesian school: A case study of Bogor, West Java.
Asia Pacific Journal of Education. Vol. 32, No. 3, September 2012.

Distance education and the complexity of accessing the Internets. Open Learning: The Journal of
Open and Distance Learning Vol. 23, No. 2, June 2008.

Roles of distance education in the implementation of the right to education in Indonesia: analysis
and lessons learnt from the sociological, political and economic points of view. Open Praxis.
2008

International Seminar

The problems of using Internet as a learning support media in open and distance education.
Paper presented on CIES Conference, New Orleans, Alabama, 2013.

155
Do increase in girl’s education attainment actually change their status in the society?-the case in
Indonesia. Paper presented on CIES Conference, Montreal, Canada. 2011.

Training rural multigrade teachers in Indonesia from distance: Bogor, West Java. Paper
presented on CIES Conference, Chicago, Illinois. 2010.

Partnership, public relations, and institutional development: The way Universitas Terbuka
attains center of excellence in Asia by 2010 and the World by 2020. Paper presented on
SEAMOLEC International Seminar, Jakarta. 2006

Distance education for sustainable development: Lessons learned from Indonesia. Paper
presented on SEAMOLEC International Seminar, Jakarta. 2006

Does technology make any difference for students?- A study of distance students’ daily life and
their use of technology? Paper presented on Asian Association Open University Conference.
Shanghai, China. 2004.

Partnership through joint program? The Universitas Terbuka maintaining mutual recognation.
Paper presented on Asian Association Open University Conference. Shanghai, China. 2004.

156