Information & Management 43 (2006) 778–793

www.elsevier.com/locate/im

Separating the effects of knowledge construction and group

collaboration in learning outcomes of web-based courses
Raquel Benbunan-Fich a,*, J.B. Arbaugh b,1
a
Zicklin School of Business, Box B11-220, Baruch College, CUNY, New York, NY 10010, United States
b
College of Business Administration, University of Wisconsin Oshkosh, Oshkosh, WI 54901, United States
Received 17 May 2005; accepted 8 September 2005

Abstract
This research investigates the effects of knowledge construction and group collaboration in outcomes of MBA courses entirely
delivered on the web. The data comes from a sample of courses conducted from summer 2000 through summer 2002. The results
indicate that students achieve higher perceptions of learning in courses where knowledge is transmitted through the system, as
opposed to constructed by the students, and when students are engaged in collaborative assignments. In terms of final grades, our
findings show that the absence of both factors (knowledge construction and group collaboration) has a detrimental effect on student
performance. Students earn on average better grades in courses where either one of these factors is present. Contrary to our
expectations, the presence of both factors does not have positive synergistic effects on student performance. These findings suggest
that the success of web-based courses entirely delivered online depends upon providing collaborative learning activities, and/or
challenging environments where the students are responsible for the creation of their own knowledge.
# 2006 Elsevier B.V. All rights reserved.

Keywords: Knowledge acquisition; Knowledge construction; Group collaboration; Learning outcomes; Technology-mediated learning; Web-
based courses

1. Introduction [19,23]. However, in order to fully understand the

The growing popularity of web-based courses at all
levels has prompted a multitude of research efforts to
test which online course design and delivery strategies
are most successful. Based on evidence provided by
multi-course/multi-section studies and single course
studies comparing online and traditional classrooms,
the consensus seems to be that collaborative approaches
are better in online courses than individualistic methods
* Corresponding author. Tel.: +1 646 312 3375.
E-mail addresses: Raquel_BenbunanFich@baruch.cuny.edu
(R. Benbunan-Fich), arbaugh@uwosh.edu (J.B. Arbaugh).
1
Tel.: +1 920 424 7189.
drivers of better outcomes, we need to make a
distinction between how the knowledge is delivered
in a web-based environment and whether the learning is
expected to occur in terms of individual activities or as a
result of group-based processes.
Research in technology-mediated learning environ-
ments has received increasing attention in the
Information Systems literature in the past decade [2].
IS scholars are uniquely qualified to study how IT
influences learning, not only due to their understanding
of the technology but also because of the long tradition
in the field to study how the technology affects
individual cognitive processes and group collabora-
tion. In particular, several empirical studies have
analyzed the learning benefits of using Group Support

0378-7206/$ – see front matter # 2006 Elsevier B.V. All rights reserved.
doi:10.1016/j.im.2005.09.001
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
Systems to enhance or to replace traditional classrooms
[1,5,28], and how Group Support Systems enable
knowledge acquisition [26,27]. However, it is not
known whether the superior outcomes achieved by
members of the GSS teams are due to the capabilities of
the technology-mediated environment to support
knowledge dissemination and sharing, and/or due to
the effects of working in groups.
Our main research question is whether the knowl-
edge delivery approach or the use of collaborative
learning techniques, or the combination of both,
improves the outcomes of web-based courses. Empiri-
cal evidence to date is somewhat mixed, in part because
the knowledge construction and group collaboration
constructs have not been conceptualized and operatio-
nalized as orthogonal constructs. In addition, most of
the prior empirical research has been focused on
comparing online versus traditional courses, rather than
testing the effectiveness of different models in web-
based courses. Therefore, we seek to analyze the
separate and joint effects of knowledge construction and
group collaboration in a sample of pure online courses.
We begin by reviewing the relevant literature in
knowledge construction and group collaboration and
developing the hypotheses. Then, we describe the
research methodology, including the experimental
design, the sample and the data collection methods.
We proceed to present the results of our statistical
analyses, followed by the discussion, the limitations and
the implications. Finally, we present our conclusions
and the contributions, and outline possible areas for
future research endeavors.
2. Literature review and hypotheses
development
Research in the Information Systems literature in the
area of technology-mediated learning includes theore-
tical and empirical studies. Theoretical studies are those
that present conceptual models fitting technology
applications to various learning theories [10,29].
Empirical studies are those that compare online learning
environments to traditional classrooms [34], or inves-
tigate the use of Group Support Systems to enable
collaborative learning activities in a particular course
[1,28], or test the effectiveness of particular synchro-
nous and asynchronous online learning environments
[4,11,32,42].
Although there are some inconsistent findings,
empirical research comparing online and traditional
courses has provided evidence of no significant
differences in learning outcomes [2]. Compared to the
779
large number of online versus traditional classroom
studies, research based on larger samples of courses
entirely delivered online is limited. This prior multi-
course research has found strong evidence for the
effectiveness of collaborative learning in asynchronous
learning environments [11]. However, two issues often
overlooked in this area are the role of the learning model
[2,29,34], and the interplay between knowledge delivery
and group collaboration in online courses.
2.1. Knowledge delivery: transfer versus
construction
In order for learning to occur, the student must
acquire knowledge. Grover and Davenport [20] define
knowledge as the most valuable form of content and
describe a continuum that starts at data, encompasses
information and ends in knowledge. The main distinc-
tion between knowledge and information is that
‘‘knowledge is information possessed in the minds of
individuals’’ [3] p. 109. Therefore, from a cognitive
perspective, knowledge is the result of mental proces-
sing generated by new stimuli.2 The central concern of
technology-mediated learning environments is how to
leverage the technology and provide the most effective
stimuli to improve knowledge acquisition.
One of the key issues in the design and delivery of
web-based courses is how knowledge is disseminated
through the medium. Although the concern is not
exclusive of technology-mediated learning environ-
ments, it is perhaps critical when instructors and
students are separated from each other in terms of space
and time. A parallel concern in education [25], echoed
in some of the Information Systems research [29], is the
debate around the appropriateness of objectivist and
constructivist approaches. These two paradigms repre-
sent fundamentally different conceptions of knowledge.
Objectivism is based on the dissemination of a stable
body of knowledge while constructivism is based on
knowledge construction. A useful analogy to under-
stand the difference between these two approaches
consists of viewing objectivism as the delivery of a
complete set of concepts that needs to be learned (for
example, a page of written text), and constructivism as
2
Alavi and Leidner [3] argue that when knowledge is articulated
becomes information and when information is cognitively processed
becomes knowledge. However, as we will discuss below, in a tech-
nology-mediated learning environment, information for instructional
purposes is highly dependent on the assumptions about the nature of
knowledge. Therefore, in this paper, we will refer to instructional
content as knowledge.
780 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
the conveyance of these concepts in pieces from
multiple sources that each learner puts together or
constructs by him/herself.
The objectivist model consists of transferring
knowledge from the professor to the students and
allowing each student to learn independently. The main
assumption in this approach is that there is a unique and
objective body of knowledge representing the world that
can be articulated and communicated to the students. As
a result, the instructor controls material and pace of
learning, and the lecture is the typical mode of
instruction [25]. Direct knowledge transfer may be
the most appropriate model for subject matters based on
factual, technical or procedural knowledge [29].
However, some critics of this approach argue that the
quality of learning is impoverished when students are
passive recipients of content that is delivered to them.
In contrast, the constructivist model assumes that
knowledge is created or constructed by every learner.
Instead of an external objective reality, the mind
produces its own conception [25]. Within constructi-
vism, there are two different schools of thought:
individual, which assumes that knowledge construction
occurs in the student’s head and social, based on the
premise that knowledge is socially constructed in the
world through interactions with others [24]. The
common element of these two views is the idea that
knowledge is constructed. In addition to emphasizing
knowledge construction, instead of knowledge transfer,
constructivist environments provide multiple represen-
tations of reality and encourage thoughtful reflection
based on personal experience.
The distinction between the instructional and self-
paced exploration approaches has also been investi-
gated in the training literature. Instruction-based
training, typically provided through lectures or discrete
packets of information, is a deductive approach where
learners are presented with the content that needs to be
learned in its final form [15,16,39]. However, explora-
tion-based learning is inductive, as learners have to
discover or construct what needs to be learned by
interacting with the content, which is usually presented
in the form of specific examples. Some researchers have
suggested that exploration is more effective than
instruction-based training in promoting the integration
of new knowledge with previous knowledge [12,15,37].
2.2. Group collaboration
A key decision in the design and delivery of web-
based courses is the extent to which students will learn in
isolation or through interactions with their classmates.
Collaborative activities allow learners greater opportu-
nities for increased social presence and a greater sense of
online community, both of which have been associated
with positive online course outcomes [36]. When
working with peers instead of alone (or just with the
instructor), anxiety and uncertainty are reduced as
learners communicate with their teammates and find
their way together through complex or new tasks [21].
In addition to improving the socio-emotional
climate, collaborative activities provide opportunities
for enhanced learning. Students learn more when they
participate in group endeavors because of specific
mechanisms that affect cognitive processes. These
mechanisms include resolution of conflicts or disagree-
ments in group discussions, internalization of explana-
tions provided by more knowledgeable peers and self-
explanation effects because explaining to others
clarifies one’s own understanding [4,11,31,34,41]. At
the same time, working in virtual teams allows students
to reflect on the written contributions of others and
encourages them to elaborate their thoughts before
expressing them in writing [21]. The success of group
collaboration in technology-mediated environments
depends to a great extent on the effectiveness of the
group of learners. Problems such as lack of commitment
by individuals, insufficient knowledge about the
activities or coordination problems may reduce the
potential learning advantages of working in groups.
Notwithstanding these potential disadvantages, the use
of collaborative activities in web-based courses is
particularly important, because the sense of community
and the typical classroom cues are often absent in this
kind of environment.
2.3. Instructional strategies and the role of
instructors
The two learning models (objectivism and con-
structivism) lead to fundamentally different choices
regarding methods of knowledge delivery and assess-
ment of knowledge acquisition. For example, con-
structivist instructors tend to focus on projects requiring
solutions to problems instead of emphasizing one-way
instruction, which is the core of objectivist approaches.
While objectivists focus on delivering sequences of
subject-matter topics, constructivists emphasize authen-
tic tasks in a meaningful context rather than abstract
instruction out of context [30]. Constructivist methods
require technological platforms that provide access to
content in a non-linear or non-structured way, and
learning tools such as databases, conceptual models,
simulations and hypermedia [29,30]. As a result, the
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
role of the instructor changes from being the main
source of knowledge and answers, to being a coach or
facilitator in the students’ construction of their own
knowledge.
Unlike constructivism, which is a learner-centered
approach where students control the pace of instruction,
objectivism is an instructor-centered approach, where
students are primarily expected to learn from the
instructor [15]. The role of instructors in a constructivist
environment shifts from conveying concepts to raising
provocative questions to motivate the students’ search
for answers, a kind of ‘‘Digital Socrates’’ [14].
However, in objectivist web-based courses, students
are exposed to frequent instructional interactions
enacted by the instructor, who is in control of the
material and pace of learning [29]. Students in
objectivist courses are more likely to generate questions
when they do not understand the material and contact
their instructors in their search for answers or
clarification. This dependence on the instructor as the
main source of knowledge and advice is likely to
generate higher levels of perceived student–instructor
communication for students in objectivist courses than
for students in constructivist environments.
Collaborative courses are designed around constant
and frequent interactions among learners, where most of
the communication takes place. These exchanges can
occur in public discussion forums, open to all the
students enrolled in a class, or in smaller group sessions,
where only students of the same team can participate
[23]. The role of the instructor in collaborative
environments resembles that of the facilitator that
raises questions to engage students in a dialog among
themselves and periodically steps in to synthesize key
ideas from group discussions [19]. In contrast, students
in individual courses are expected to work on their own.
Although contact with other students probably takes
place, it is neither required nor necessary. Therefore,
students in collaborative courses are likely to experi-
ence higher levels of learner–learner communication
than students in individually oriented courses.
2.4. Learning effects
Learning involves the acquisition of knowledge,
which is evidenced in changes of behavior leading to
better performance [2]. Since knowledge acquisition is
not directly observable, the learning effects can be
manifested in terms of improved perceptions of learning
or better performance in assessment and evaluation
activities. Hereafter, we refer to these two constructs as
learning outcomes.
781
The premise of constructivism is that learning is an
active process of constructing rather than acquiring
pre-existing knowledge. In this view, knowledge is
not just a mental state; it is an experienced relation
of things, and learning is the result of a process of
inquiry at the individual and/or at the social level [24].
Leidner and Jarvenpaa [29] p. 267 propose that
‘‘individuals are assumed to learn better when they are
forced to discover things themselves rather than when
they are told or instructed’’. Thus, the construction of
knowledge should foster a deeper understanding and
better learning in students than the mere transfer
from the instructor. Based on this conjectures, we
hypothesize:
H1. Students engaged in knowledge-construction web-
based courses will achieve better learning outcomes
than students in web-based courses where knowledge is
directly transferred from the instructor.
In web-based collaborative courses, students are
engaged in intellectual exchanges with their peers that
require verbalization of their current understanding of
the material and the articulation of their cognitive
processes. This articulation can be motivated by the
need to explain concepts to peers, or to make explicit
their problem-solving strategies [34]. In addition to
these cognitive benefits, working with peers also
provides socio-emotional advantages [21,41], both of
which are conducive to better learning outcomes. In
contrast, students in individual-based courses have
limited possibilities for developing these higher-order
cognitive processes associated with group interaction
[13,22]. Therefore, we hypothesize:
H2. Students in web-based courses structured around
collaborative activities will achieve better learning out-
comes than students in individualized web-based
courses.
Based on the theorized effectiveness of constructi-
vism and the learning advantages of working in
groups, the combination of knowledge construction
with group collaboration is expected to have positive
synergistic effects on learning outcomes because it
requires students to construct knowledge by interact-
ing with others. Thus, learning is the result of a
process of inquiry where meaning is socially
constructed [24]. The potential for achieving a deeper
understanding and more learning when knowledge is
constructed [29], in conjunction with the advantages
of verbalization and articulation when students are
learning from others in a collaborative environment
782 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
[11], are expected to result in the best levels of
learning. Thus, we hypothesize:
H3. Students in collaborative constructivist web-based
courses will achieve the best learning outcomes.
3. Research methods
3.1. Experimental design
The combination of knowledge construction with the
presence of group collaboration produces a 2  2
framework with four different categories: transfer-
individual, transfer-group, construction-individual and
construction-group (see Fig. 1). These categories
describe four possible web-based learning environ-
ments.
A transfer-individual course is based on the direct
knowledge transfer from the instructor to the students,
typically via prepared lectures posted on the course web
site weekly or by modules. Students in these courses are
required to learn and master the material individually
and demonstrate their assimilation of pre-ordained
knowledge by working individually in the completion
of assignments or exams.
A transfer-group course is also based in a one-way
transmission of knowledge from the instructor to the
students through the use of lecture notes. However, this
type of course requires the students to work together and
to co-operate in the completion of group projects and
other group-based assignments. The mastery of the
material is accomplished not only by individually
assimilating content, but also by working in group
assignments that reinforce the learning process.
A construction-individual course provides an envir-
onment where students construct their own knowledge
independently, by actively interacting with the subject
matter. Instead of receiving modules of pre-ordained
knowledge (as in transfer courses), students gather
Fig. 1. Experimental design.
materials and combine information from different
sources. This process of investigation and discovery,
usually presented in the form of a complex project or
case study, allows the student to understand the subject
matter and create knowledge.
A construction-group course is also based on the
premise that students search for meanings instead of
receiving packaged lectures. Rather than engaging in
individual investigations, students in this type of course
work together in small groups to discuss cases and/or to
collaborate on projects. This kind of environment
requires students to interact with others in the knowl-
edge construction process.
The goal of the transfer-group approach is to allow
students to work together on problems with objectively
correct answers that require them to express the specific
facts and concepts they have memorized or learned.
Accordingly, the main difference between the transfer-
group approach and the construction-group category is
that in the former category students do not create new
meaning, but use knowledge previously transmitted (or
directly made available) by the instructor. It may be
argued the transfer-group approach requires students to
search for pre-ordained knowledge and therefore should
be considered a form of constructivism. Strictly
speaking, however, students in these kinds of courses
are not required to create new meaning or build
knowledge in order to solve the problem, but rather
reproduce existing concepts, principles or methods. In
doing so, students primarily learn with others and
experience some of the cognitive benefits of group
work.
The novelty of this research framework is the
combination of instructor-led knowledge transfer with
the use of collaborative activities. Typically, research
in technology-mediated learning environments associ-
ates group collaboration with knowledge construction.
This may not be always the case, however. Even when
direct instruction is used as the knowledge transfer
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
approach, students may be engaged collaborative
learning activities that require them to demonstrate
what they have learned [24]. The addition of group
collaboration to traditional knowledge transfer meth-
ods overcomes one of the major drawbacks of this
approach because students are no longer passive
recipients of content.
3.2. Sample
The experimental framework allows us to analyze
the effects of knowledge construction and group
collaboration on learning outcomes. To test our
hypotheses regarding the separate and joint effects of
these two factors, we conducted a field experiment in an
MBA program of a Midwestern U.S. university. We
surveyed 40 online class sections from summer 2000
through summer 2002. The courses covered a range of
topics including Strategy, Organizational Behavior,
Project Management, International Business, Human
Resources, Finance, Accounting, Management Infor-
mation Systems, E-Commerce, and Special Topics in
Management. In most cases, more than one section per
course was offered during the time frame of the study
and included in the sample.
These courses were delivered entirely online. Six of
them were administered via its own web site using
Lotus LearningSpace software and the others were
Table 1
Courses, class sections, enrollments, and respondents
Course
Foundations of finance
International financial management and investments
Managerial accounting/cost management
Management information systems integration
Management information systems analysis and design
Managerial problem solving
Human resources management
International business
Organizational foundations
Organizational leadership and change
Personal and professional development
Strategic management
Marketing strategy
E-commerce to E-business
Classic and contemporary literature in business
Planning for management in the future
Environmental management
Introduction to project management
Advanced topics in project management
Project execution and control
Business environments: law, regulation, and ethics
Total
783
delivered using the Blackboard CourseInfo platform.
Fifteen different instructors taught the courses. Thirty-
two courses had an on-site meeting to begin the course
while seven had an on-site meeting at the end of the
course to administer the final exam. The remaining
section did not have any face-to-face meetings on-
campus. Class section enrollments ranged from 9 to 35
students. Table 1 provides more detail about the
courses, the number of sections, enrollment information
and respondents by course.
3.3. Data collection methods
Three data sources were used in this study: students’
responses to a questionnaire, instructors’ interviews to
determine course characteristics, and secondary sources
to identify course design indicators. Data from students
was collected in a post-course survey. Students enrolled
in courses that had a final on-campus meeting
completed the questionnaire in the last class. Students
who did not come to campus for a final meeting were
sent the survey via e-mail. The non-responding students
were mailed a copy of the questionnaire. The usable
response rate was 66% (579 of 872). For each of the
respondents, we also collected the final grades obtained
in the course.
To classify the sections in the study on the basis of
the knowledge construction/group collaboration frame-
No. of class sections Total enrollment No. of respondents
2 39 20
1 14 10
2 52 38
1 31 22
1 16 6
1 21 20
3 82 56
4 98 62
3 57 30
2 46 34
2 52 42
3 69 48
1 26 14
1 23 8
3 63 43
2 49 34
1 16 11
3 79 46
2 25 13
1 12 8
1 22 14
40 872 579
784 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
work, we used a two-step process. First, we identified a
set of four course design indicators, namely lectures,
small group discussion forums, projects and exams. By
reviewing the corresponding course web sites, we
determined how many courses used lecture notes (in the
form of PowerPoint and in the form of audio lectures),
and how many used small discussion forums. We also
determined by reviewing the course syllabi the use of
projects (individual and/or in groups) and the use of
exams, as assessment techniques. Based on these
indicators, we established an initial course classifica-
tion. However, there were a number of courses whose
classification was not clear because the information
from secondary sources was not definitive.
Therefore, as a second step, we conducted semi-
structured interviews with all the instructors. These
interviews probed their knowledge delivery approach
and assessed whether the predominant type of learning
activity in their courses was individual or group-based.
To learn about the instructional methods, faculty were
asked whether their courses were focused primarily on
fact/concept dissemination via web-based lectures, or
based on knowledge construction by students. To
investigate the presence of the group-learning para-
digm, faculty were also asked to compare the proportion
of individual and group learning activities in their
courses and choose the predominant method. The
interview protocol is presented in Appendix I. We
adopted a final course classification according to the
information provided by the instructors.
3.4. Operationalization of variables
Courses in the sample were classified according to the
knowledge delivery approach (transfer versus construc-
tion) and the collaboration dimension (individual versus
group). These two dimensions were operationalized with
two categorical variables: knowledge construction, KC
(0 for transfer and 1 for constructed) and group
collaboration, GC (0 for individual and 1 for groups).
Based on the instructors’ interview responses, 4 sections
were classified as transfer-individual, 14 sections were
categorized as transfer-group, 9 sections were placed in
construction-individual and 13 sections were categorized
as construction-group. In some cases, two sections of the
same course were classified in different cells of the
framework, based on the answers provided by each of the
instructors. For example, two sections of Strategic
Management were classified in the construction-group
category, while the third was placed in the transfer-group
cell. Similarly, the three sections of human resources
management were assigned to different categories
(transfer-group, construction-individual, and construc-
tion-group) according to the instructors’ responses.
To study the nature of the interaction that took place
in the web-based courses, several items were included
in the post-course student questionnaire. Student
perceptions of learner–instructor communication were
measured using seven items and student perceptions of
learner–learner communication were measured using
four items. Both of these scales were adapted from
instruments used by [38].
The main dependent variables used in this study are
perceived student learning and actual grades. The
perceived learning scale was originally developed by
[1,22], and adapted for this study. Perceived learning
measures the extent to which students believe they
have learned in the course. Higher levels of perceived
learning are likely to result in higher student
satisfaction, which in turn should translate into higher
ratings for the course, often used as a measure of
success [28].
Factor loadings and reliability results for the learner–
instructor, learner–learner and perceived learning scales
are shown in Table 2. All the items measuring each
perceptual variable clearly loaded onto its correspond-
ing factor. These loadings range from 0.64 to 0.86
indicating a strong correlation between each of the
items and the variable it measured. In addition, there are
no cross-loadings greater than 0.45. This analysis
provides evidence of good convergent and discriminant
validity.
The reliability coefficients of the scales are 0.92,
0.85 and 0.94 for the learner–instructor, learner–learner
and learning perception scales, respectively. In addition,
except for one item in the learner–learner scale, all the
other single item coefficients are higher than 0.80.
Table 3 shows the reliabilities of each item with the
scale and the scale Cronbach’s a coefficients. These
reliabilities are high and well above the cutoff rate of 0.7
recommended by [33].
The second dependent variable is the final grade that
students earned in the courses. This is a measure of
objective performance, and it is an indicator of actual
learning. Earlier research indicates that self-reported
learning is often inconsistent with objective perfor-
mance measured by grades [28]. While learning
perception indicates the extent to which students feel
they learned in the course, actual learning measures the
degree to which they actually learned based on the
performance assessment methods used by the instruc-
tors. Therefore, the use of final grades as a dependent
variable incorporates a more objective measure of
knowledge acquisition.
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793 785

Table 2
Factor loadings and cross loadings of the indexes
Learner–instructor Learner–learner Perceived learning
Learner–instructor
The instructor frequently offered opinions to students 0.80 0.04 0.24
The instructor frequently asked the students questions 0.78 0.19 0.24
The instructor frequently attempted to elicit student interaction 0.78 0.25 0.29
Interaction between the instructor and the class was high 0.76 0.20 0.23
Students often asked the instructor questions 0.74 0.03 0.16
Students often stated their opinions to the instructor 0.71 0.22 0.21
In general, the instructor was effective in motivating the 0.64 0.33 0.39
students to interact in this course
Learner–learner (all reverse scored)
Students seldom answered each other’s questions 0.17 0.79 0.11
In this class, students seldom stated their opinions to each other 0.15 0.78 0.15
There was little interaction between students 0.28 0.71 0.16
The students seldom asked each other questions 0.29 0.69 0.18
Perceived learning
Good understanding of basic concepts 0.22 0.11 0.86
Identified central issues in field 0.21 0.11 0.85
Course quality compared favorably to other courses 0.24 0.07 0.82
Improved ability to integrate facts 0.20 0.14 0.80
Learned to see relationships between ideas 0.25 0.14 0.77
Improved ability to communicate clearly 0.33 0.12 0.68
Course served my needs well 0.27 0.18 0.60
Satisfied with the course 0.31 0.15 0.58
High quality class discussions 0.38 0.43 0.47

3.5. Statistical analyses 4. Results

The unit of data collection and the unit of analysis
for testing our hypotheses was the individual student.
However, since the students were enrolled in sections
when they received the treatment (i.e. experienced the
web-based course), it is likely that the interaction
between students and their common experiences for
being in the same class will influence their percep-
tions. This influence or class level effect implies that
the error terms contain a systematic component [35].
The violation of independence is one of the main
reasons for not using traditional regression models.
Ignoring the group effect could lead us to falsely
reject the null hypothesis (no effect of treatment)
because the degrees of freedom are artificially
increased [40].
To take into account that students are nested units
within classes, and these sections are in turn, grouped in
different treatments, we used multilevel analysis of
variance, also known as hierarchical analysis of
variance (HANOVA). This technique allows studying
the effects of course characteristics on individual level
outcomes, while controlling for the fact that students
received the treatment in a group setting or web-based
course [18].
4.1. Course classification and manipulation checks
We validated the course classification derived from
the instructors’ interviews, using the four course design
indicators (lectures, discussion forum, projects and
exams) and our own original classification for each
course. First, we coded each indicator with dummy
variables and analyzed these variables vis-à-vis the
course classification. According to the premises of the
knowledge delivery approaches, there should be a
preference for lectures in direct knowledge transfer
courses, and a preference for projects in knowledge
construction courses. In addition, individually oriented
courses should be structured in terms of individual
projects and exams, instead of group-based exercises. In
contrast, collaborative courses should use group
projects and small discussion forums to complement
(or perhaps replace) individual assessment methods.
Consistent with the premises of each approach, we
noticed that none of the transfer-individual courses in
our sample used team projects, while all of the
collaborative sections used small group discussion.
We also tested for statistical differences between the
proportions of courses with each of these indicators,
786 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793

Table 3
Scale reliability
Scale Items Cronbach’s a coefficient
Learner–instructor 0.92
The instructor frequently offered opinions to students 0.91
The instructor frequently asked the students questions 0.91
The instructor frequently attempted to elicit student interaction 0.90
Interaction between the instructor and the class was high 0.91
Students often asked the instructor questions 0.92
Students often stated their opinions to the instructor 0.91
In general, the instructor was effective in motivating the students to interact in this course 0.91
Learner–learner 0.85
Students seldom answered each other’s questions 0.82
In this class, students seldom stated their opinions to each other 0.82
There was little interaction between students 0.79
The students seldom asked each other questions 0.81
Perceived learning 0.94
Good understanding of basic concepts 0.93
Identified central issues in field 0.93
Course quality compared favorably to other courses 0.93
Improved ability to integrate facts 0.93
Learned to see relationships between ideas 0.93
Improved ability to communicate clearly 0.93
Course served my needs well 0.93
Satisfied with the course 0.93
High quality class discussions 0.94

using the course-section as the unit of analysis. Table 4 between construction and transfer courses. In terms of
presents these results along with the proportion of individual and collaborative assignments, there is a
courses in each condition where the dummy variable for marginally significant difference between the propor-
the indicator is 1. tion of individual sections using exams when compared
When compared to knowledge construction sections, to collaborative courses (54% versus 30%, z-value 1.48;
a larger proportion of knowledge transfer courses use p < 0.10). Also marginally significant is the proportion
lecture notes (z-value = 2.40, p < 0.01), and audio of collaborative courses using group projects in
lectures (z-value = 2.08; p < 0.05) as vehicles for comparison to individual-based courses (37% versus
disseminating knowledge. No significant differences 15%; z-value = 1.4; p < 0.1). Finally, the preference for
are found in the use of individual or group projects small discussion forums is more noticeable in group-
based courses than in individualistic ones. Every
collaborative section used small group discussion,
Table 4 while only 62% of the individual sections did. Thus,
Tests of differences in proportions of course design indicators
when analyzed separately, these course design indica-
Knowledge construction Transfer Construction z-Value tors based on syllabi information and web site
course classification (%) (%) inspection, are consistent with the information provided
Lecture notes 100 73 2.40** by the instructors regarding the nature of their courses.
Audio lectures 78 45 2.08* For a second validation, we compared our original
Individual projects 89 95 0.78 author-derived course classification based on the course
Group projects 22 36 0.39
design indicators with the final one determined from the
Group collaboration Individual Collaborative z-Value instructors’ interviews. The level of agreement between
course classification the two classifications is 0.8 (out of the 40 sections, 8
Individual projects 85 96 1.31
were placed in different cells of the framework). The
Exams 54 30 1.48$ disagreements were solved in favor of the instructor-
Group projects 15 37 1.4$ provided classification. Taken together, these analyses
Small groups 62 100 3.44*** validate the results of the interviews and increase the
Significance levels: *p < 0.05; **
p < 0.01; ***
p < 0.001; $p < 0.10. reliability of our course classification.
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793 787

As an additional manipulation check, we analyzed two means is small, this result is statistically significant
the learner–instructor and learner–learner communica- due to the separation of the nested factor from the error
tion scales using a hierarchical ANOVA model with two term in our analyses. When the systematic component
main factors (knowledge construction and group created by the class level effect is removed from the
collaboration), their interaction and one nested factor residual error, the magnitude of the error term is
due to the course/section. According to the premises of reduced. If the nested factor is treated as the error term,
knowledge delivery, higher levels of learner–instructor the significance of knowledge construction is p = 0.06,
communication should take place in knowledge transfer as indicated in the footnote of the table. Both analyses
courses than in knowledge construction courses, and suggest that the perception of learner–instructor
higher levels of learner–learner communication should communication is higher for students in knowledge
occur in collaborative courses when compared to transfer courses than for students in knowledge
individually oriented courses. Table 5 shows the means construction courses.
by condition and the results of these tests. The group collaboration factor in the learner–learner
The knowledge construction factor is significant in HANOVA model is significant (F = 112.11; p < 0.001)
the learner–instructor HANOVA model (F = 34.15; and the marginal means of the collaborative versus
p < 0.001), and the marginal means indicate that individual students are in the expected direction (5.33
students in courses where knowledge is transferred versus 4.29). The significance of this factor does not
(as opposed to constructed) report higher levels of change when the nested factor is used as the error term.
communication with their instructor (4.95 versus 4.85, Thus, students in group-based courses report higher
respectively). Although the difference between these perceptions of communication with their classmates,
than students in individually oriented courses.
Table 5
HANOVA results on interaction variables 4.2. Test of hypotheses
HANOVA results Learner–instructor Leaner–learner
communication, Fa communication, Fb Our hypotheses predicted relationships between the
Model 11.58 ***
9.50 *** main factors (knowledge construction and group
Knowledge construction 34.15*** 39.81*** collaboration) and the learning outcomes (H1 and
(KC) H2, respectively) and an interaction effect (H3). We test
Group collaboration 81.04*** 112.11*** each hypothesis using both learning perception and
(GC)
grades as dependent variables.3 The results of these tests
Interaction (KC  GG) 73.22*** 30.91***
Nested factor 9.70*** 6.37*** are shown in Table 6. We report in the table the analysis
(course–section) when the nested factor (course/section) is separated
R2 0.46 0.41 from the error term and include footnotes to show the
results when the nested factor is used as the error term.
n Means Means For both dependent variables, hierarchical MANOVA
Cell means model results on the nested factor (course/section) are
Transfer/individual 36 3.45 (1.23) 3.31 (1.37) significant at 0.1%. These results indicate the presence
Transfer/collaborative 191 5.23 (1.04) 5.30 (1.15)
of group level effects due to the particular character-
Construction/individual 128 4.86 (1.27) 4.56 (1.61)
Construction/collaborative 221 4.84 (1.31) 5.35 (1.34) istics of each section. The inclusion of the nested factor
controls for the variations in the particular implementa-
Marginal means
Transfer 227 4.95 (1.26) 4.99 (1.39)
tions of each course/section.
Construction 349 4.85 (1.29) 5.06 (1.38) According to H1, students engaged in knowledge
Individual 164 4.55 (1.39) 4.29 (1.64) construction were expected to achieve better learning
Collaborative 412 5.02 (1.21) 5.33 (1.14) outcomes than students in courses where knowledge is
Notes: Standard deviations in parentheses. Significance levels:
*
p < 0.05; **p < 0.01; ***p < 0.001.
a 3
If the nested factor (course–section) is used as the error term, The correlation between the two dependent variables (grades and
instead of the residual error, the significance of knowledge construc- perception of learning) is 0.13, and this correlation is significant at
tion is 6%. The other factors retain the same levels of significance. 1%. Its low level confirms that objective and perceived learning are
b
If the nested factor (course–section) is used as the error term in this two different aspects of learning. However, since both dependent
analysis, instead of the residual error, group collaboration is signifi- variables are significantly correlated, we use a hierarchical MANOVA
cant at 1% and knowledge construction and the interaction factor are models. The results of separate HANOVA analyses are similar to
each significant at 5%. those reported here.
788 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793

Table 6
Hierarchical MANOVA results on perceived learning and grades
HANOVA results Wilks’ l Perceived Grade, Fb
learning, Fa
Model 3.71*** 4.91***
Knowledge construction 2.08 4.25* 14.84***
(KC)
Group collaboration (GC) 6.17** 32.80*** 11.46***
Interaction (KC  GG) 3.71* 18.08*** 9.87**
Nested factor 2.97*** 4.63*** Fig. 2. Interaction effects on learning perception.
(course–section)
R2 0.21 0.26
second dependent variable, the marginal means of
n Perceived learning Grade
grades in collaborative courses are higher than those
means means
in the individual courses (3.73 versus 3.66) and the
Cell means group collaboration factor is significant (F = 11.46;
Transfer/individual 36 4.01 (1.94) 3.38 (0.83)
p < 0.001). Thus, H2 is supported for both dependent
Transfer/collaborative 190 5.20 (0.97) 3.72 (0.52)
Construction/individual 128 4.63 (1.39) 3.75 (0.38) variables.
Construction/collaborative 221 4.86 (1.31) 3.74 (0.38) The interaction effect hypothesis (H3) predicted that
Marginal means
students in construction-group courses were expected to
Transfer 226 5.01 (1.25) 3.66 (0.59) achieve the highest levels of perceived learning across
Construction 349 4.77 (1.34) 3.74 (0.38) conditions. With respect to learning perception, the
Individual 164 4.49 (1.54) 3.66 (0.53) interaction effect of knowledge construction and group
Collaborative 411 5.01 (1.17) 3.73 (0.45) collaboration is significant (F = 18.08; p < 0.001), but
Notes: Standard deviations in parentheses. Significance levels: not in the expected direction. The means by condition
*
p < 0.05; **p < 0.01; ***p < 0.001. indicate that the highest perceptions of learning
a
If the nested factor is used as the error term in the learning
perception model, group collaboration is significant at 1% while
correspond to students in transfer-group courses
knowledge construction and the interaction of both factors are sig- (5.20). Fig. 2 presents graphically the interaction
nificant at 5%. between knowledge construction and group collabora-
b
If the nested factor is used as the error term in the model with tion on learning perception. Since the line segments
grades as dependent variable, knowledge construction is marginally
cross each other, there is a disordinal interaction. The
significant at 10%.
rank order of the effectiveness of the different
conditions on learning perception is: transfer-group,
transferred. The hierarchical MANOVA model result construction-group, construction-individual and trans-
for knowledge construction on learning perception is fer-individual. The combination of knowledge con-
significant (F = 4.25; p < 0.05), but the marginal means struction and group collaboration does not produce the
indicate that students in knowledge construction highest means. Therefore, H3 is not supported for
courses report on average lower learning perception learning perception.
than students in knowledge transfer courses (4.77 versus In terms of performance (grades), the lowest
5.01, respectively). Thus, H1 is not supported for averages are found in students enrolled in transfer-
learning perception. In the test of H1 using grades as a individual courses (means 3.38), whereas in the other
dependent variable, the results are opposite. Students in conditions the averages are in the 3.7 range. The
courses based on knowledge construction achieve significance of the interaction factor (F = 9.87;
higher grades than their counterparts in transfer courses p < 0.01) indicates that the presence of either knowl-
(means 3.74 versus 3.66; F = 14.84; p < 0.001). edge construction or group collaboration has a positive
Therefore, H1 is supported for grades. effect on student grades. However, the presence of both
The second hypothesis predicted better learning does not seem to have any positive synergistic effect, as
outcomes for collaborative courses when compared to hypothesized (H3). Fig. 3 shows graphically the
individualistic ones. For our first dependent variable, interaction effects of the two factors on final grades.
learning perception, the marginal means are higher for Knowledge construction is superior to knowledge
collaborative courses than for non-collaborative courses transfer but not by the same amount for the
(5.01 versus 4.49), and the group collaboration factor is individual-oriented and group-oriented courses. In fact,
significant (F = 32.80; p < 0.001). Likewise, for our the final grades for collaborative courses are very
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
Fig. 3. Interaction effects on grades.
similar, regardless of the knowledge delivery approach.
Since the combination of collaboration and construction
does not produce the highest scores, H3 is not supported
for grades.
5. Discussion of results
The findings for the learner–instructor and learner–
learner communication manipulation checks suggest
that the courses were delivered as intended, and as
reported by the instructors in the interviews. These
student level perceptions provide additional support to
our course classification variables determined from the
information provided by the instructors and the analysis
of course design indicators. In addition to serving as
manipulation checks, these measures are also evidence
of the communication dynamics in each category of the
experimental framework. One noteworthy finding in
this regard is the low levels of perceived learner–
instructor communication in transfer-individual
courses. This suggests that the delivery of knowledge
in these sections is mostly achieved via a set of
packaged materials such as lecture notes and audio
lectures that is not complemented with live participation
or explanations from the instructor.
In terms of learning perception, our results indicate
that the most successful combination for the design of
web-based courses is found in the transfer-group
category. Courses in which knowledge is directly
transmitted from the instructor to the students and
collaborative approaches are used as the primary learning
mode scored significantly higher in perceived learning
than did courses that were conducted using other
combinations of knowledge delivery and collaboration.
This finding confirms previous conceptual and empirical
work supporting the importance of collaborative learning
in web-based learning [23,29]. It is interesting to note that
transfer-group courses are where the higher perceptions
of learner–instructor and learner–learner communication
occur. These results suggest that students not only
communicate more with each other, but also with the
789
instructor, because of his/her dual role as knowledge
expert who delivers the material and group facilitator
who guides the student teams. Because of this double
duty, the level of perceived learner–instructor commu-
nication is significantly higher in transfer-group than in
transfer-individual sections.
In terms of knowledge acquisition measured by final
grades, our results indicate that the presence of either
group collaboration or knowledge construction sig-
nificantly improves the students’ final grades. This
finding also supports the importance of collaborative
approaches documented by previous research, and
suggests that knowledge construction is also beneficial
for student performance. The combination of both,
however, does not produce positive synergistic effects
on grades. A possible explanation is that better learning
can be equally achieved by constructing knowledge or
by articulating knowledge to others, and that these
two mechanisms substantially overlap when used in
combination.
One common finding for both dependent variables is
that students in transfer-individual courses exhibited
the lowest perceptions of learning and the lowest grades
across all conditions. The number of sections classified
in this category (only four) and the learning outcomes in
these courses indicate that this mode is neither the most
preferred by instructors nor the most successful in terms
of learning in web-based environments. Given that the
perceptions of instructor–learner communication are
the lowest for this category, it seems that instructors in
this mode used the technology as a replacement for their
role as knowledge communicators. To achieve better
learning outcomes, particularly for graduate manage-
ment education, participants in online courses need to
collaborate with their counterparts in virtual teams and/
or to be engaged in the construction of their own
knowledge [8,37].
Taken together, the results on both dependent
variables show an interesting paradox. Students who
are in knowledge construction courses do not perceive
that they have learned as much as students in knowledge
transfer courses. However, their final course grades
show that they learn more or better than students in
knowledge transfer sections. Because ‘‘constructive’’
learning is less structured and requires students to make
sense of the material, students may feel they have
learned less, a perception that is at odds with their actual
knowledge acquisition evidenced in their final grades. It
is also possible that due to the different nature of student
performance in graded activities in both types of
courses (for example, term papers or essays for
knowledge construction courses versus multiple choice
790 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
exams for knowledge transfer courses), performance
assessment is more subjective and results in higher
grades when students construct their own knowledge
than when students reproduce objective content. In
either case, this finding underscores that learning
perception and performance are not always consistent,
as they are measuring different aspects of the learning
experience.
In sum, by separating the effects of knowledge
construction from the effects of group collaboration, our
results indicate that collaborative approaches result in
better learning perception, even in courses where
knowledge construction is not the dominant or the most
efficient approach. In terms of grades, we found that
either one of these factors has a positive effect on
student performance. The evidence provided by this
research is stronger than previously reported results on
this area because it includes multiple courses and
compares different approaches at once. To our knowl-
edge, no other empirical study has attempted this multi-
course comparison before.
Unlike prior multi-disciplinary or multi-course
empirical studies, our results are robust to the
systematic errors introduced by the use of different
sections in each mode (group level effects). Using
hierarchical analysis of variance, we separate the effects
of the main factors (knowledge construction and group
collaboration), and their interaction from the nested
factor (course–section). This technique has been
previously used by [4,27] to control for the effects of
working in small groups when collecting and analyzing
individual level data. However, to the best of our
knowledge, we are the first to use this technique in a
multi-course study of online environments where the
nested factor is the course/section.
5.1. Limitations
There are several limitations that must be taken into
account in the interpretation of results. First, since the
hypotheses were tested in a field experiment in the
context of an MBA program rather than in a controlled
laboratory experiment, some internal validity is lost at
the expense of a greater ability to generalize the results.
Although these findings may not be exactly reproduced
in other contexts such as other kinds of programs or
university settings, the fact that we obtain significant
effects attests to the importance of the factors examined
here. As Dennis et al. [17] p. 186 point out ‘‘If the
processes of interest are still observed in an uncon-
trolled environment with all of its extraneous influences,
this bespeaks of the power of those processes.’’
Second, we did not explicitly take into account the
particular characteristics of the instructors or the subject
matters taught in these courses [7]. We used the
professors’ opinions to classify the courses, without
imposing our own bias with respect to which courses
were more suitable for knowledge construction versus
knowledge transfer, or for collaborative versus indivi-
dual activities. In fact, we classified several sections of
the same course in different cells of the framework
based on the results of the instructors’ interviews. This
added flexibility to the comparative field study and
allowed us to examine the effects of the two factors
beyond the particular characteristics of each instructor
and subject matter. However, we control for the
combined effect of subject matters and instructor
characteristics with the use of the nested factor in
our analyses.
Another limitation is that we developed our research
framework by combining two dimensions (knowledge
construction and group collaboration) that were
operationalized with categorical indicators. In our
faculty interviews, we discovered that sometimes
instructors combine both knowledge delivery models
but subscribe predominantly to one approach over the
other. Similarly, our analysis of course design indicators
revealed that instructors sometimes use both group
collaboration and individual activities. In this study, the
perceptions of learner–learner and learner–instructor
communication reported by the students confirmed the
classification obtained from the instructors’ interviews.
Follow-up studies may choose to operationalize the
knowledge delivery approaches (direct transfer versus
constructive) as continuous variables, rather than
categorical variables. This would, of course, require
an intensive and extensive validation process to ensure
the validity and reliability of these metrics when applied
to a multi-course, multi-instructor sample such as this
one.
Our dependent variables (learning perception and
final grades) may also be affected by methodological
limitations inherent to the mechanisms whereby these
measures are collected (self-reports in the case of
perceived learning) or produced (determined by the
instructors in the case of final grades). Nevertheless, the
use of both dependent variables offers a more complete
picture of the learning experience from the students’
perspective. It is possible that the discrepancy in the
findings of learning perception and final grades for
knowledge construction and knowledge transfer courses
may have resulted from a familiarity bias. Students were
probably more familiar with instructional methods than
with constructivist approaches and this familiarity may
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
have produced higher levels of perceived learning in
knowledge transfer courses, despite the fact that final
grades are in the opposite direction.
Another limitation is found in the types of measures
used in this project. Since this is a quantitative study, it
is likely that qualitative sources of data such as content
analysis of web-based interactions or students inter-
views, may add more depth to our findings [9]. We
welcome the efforts of future researchers to refine and
improve on these methods and measures in future
studies.
There are some advantages offsetting these limita-
tions. The main strength of this study is the validation of
the course classification decisions (determined from the
instructors’ interviews) with two additional methods
and data sources. First, we used course design indicators
extracted from syllabi and course web sites to ascertain
whether the courses had different structures. Second, we
also used the students’ perception of learner–instructor
and learner–learner communication from the post-test
survey to confirm whether the courses were delivered as
described in the syllabi and reported by the instructors.
This validation is equivalent to manipulation checks
usually performed in controlled experiments.
Another strong point of this study, which sets it apart
from previous empirical research in multi-course
samples, is the use of hierarchical data analysis methods
to control for course–section effects.
5.2. Implications
These findings have important implications for the
design of online courses. Some practical implications
include the importance of incorporating collaborative
learning techniques to create virtual teams of students
and foster the growth of a learning community. Group-
communication facilities, such as discussion boards and
chat rooms should be brought to the forefront of the
design and delivery of online courses, rather than being
an optional or an after-thought activity. This research
shows the importance of these group communication
facilities to improve the level of learning outcomes
achieved by students in online courses.
This study also shows the importance of developing
challenging learning environments where students can
discover and create knowledge on their own [6]. These
settings are particularly beneficial for knowledge
acquisition (measured by final grades). In light of the
increasing popularity of web-based courses in graduate
and undergraduate programs and their increasing use in
corporate training, these findings suggest that online
communities for learning or training may obtain better
791
learning outcomes when their work is structured around
collaborative activities and/or around knowledge con-
struction exercises.
There are also important implications for instructors
of online environments, identified by prior research but
presented in a new light in this study. Although their
role may be different, instructors are principal actors in
any environment [14,42]. Regardless of the knowledge
delivery approach, successful web-based environments
require active involvement from instructors. Our study
shows that in the context of web-based MBA courses,
the technology cannot be used to replace instructors, but
rather to complement their task. From the design of the
course to its delivery, instructors must create active and
engaging learning environments for their students and
closely monitor the development of these environments
while the course is in progress.
The research implications of this study are manifold.
Future multi-disciplinary studies should expand upon
the predictors of outcomes of web-based courses in two
directions – individual level variables and course level
variables – and use multilevel analysis techniques to
assess the significance of these predictors. In addition,
the knowledge construction/group collaboration frame-
work developed here can be used to examine the
effectiveness of web-based courses at the undergraduate
level and/or in other institutional contexts. Finally,
researchers should focus their efforts on the develop-
ment of alternative measures to capture the underlying
learning model implemented in each course. These are
intriguing areas for future research endeavors.
6. Conclusions
This research contributes to the IS literature in
technology-mediated learning by analyzing the effects
of knowledge construction and group collaboration in a
multi-course sample, and showing how each dimension
contributes to learning perception and learning perfor-
mance. With the use of a multi-course sampling frame,
we assess the effects of each factor separately and in
combination across a range of subject matters and
validate the commonly held assumption that collabora-
tive approaches are better than individual approaches
for online courses. Our findings also provide new
evidence of the importance of knowledge construction
approaches in web-based MBA courses, particularly in
terms of final grades.
Our results suggest that the one-way transmission of
knowledge, where students work individually with a set
of packaged materials is not the most successful
approach for web-based courses. Such courses scored
792 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
lowest on both the dependent variables included in
our study (perceived learning and objective learning).
Learners in this kind of environments do not receive
the stimulation, the reinforcement and the social
support usually found in group activities. The social
dimension of learning appears to be essential for the
success of these courses, where the sense of the
classroom may be lost in the technology-mediated
environment. Likewise, the construction of knowl-
edge seems to be essential to engage and challenge
learners that are dispersed in time and distributed in
place, particularly when they are expected to work
individually.
This study opens new possibilities for future
research, specifically in the area of multi-disciplinary
studies. Given the amount of research already con-
ducted in the field, the comparison between traditional
classrooms and online environments in the context of
single courses appears to be no longer productive. The
new research challenge is to understand how and why
some online environments are more successful than
others and derive meaningful lessons, which are
applicable across several disciplines and for a wide
range of courses. This challenge echoes the call for
more breadth and depth in technology-mediated
learning [2], and further suggests that by examining a
broader sample of courses with a deep understanding of
the different implementations of virtual environments,
these two objectives can be achieved.
Acknowledgments
We are very grateful for the useful suggestions
provided by the anonymous reviewers and the editor of
the SOS section of Information & Management, and for
the feedback offered by various colleagues at our
institutions.
Appendix A. Interview protocol for instructors
I’d like you to think about the course _ you taught in
the semester _. As you know, your course is part of
larger sample of online MBA courses that we are
examining in a research project. We are interested in
comparing the predominant teaching and learning
modes in those courses.
A.1. Regarding teaching
Which of the following would you say best
represents the PRIMARY mode of teaching in that
course?
a. Did you provide/present course material and students
subsequently demonstrated how well they acquired
that material?
b. Did you provide course material from multiple
sources and students put together and made sense of
the material to create knowledge for themselves?
A.2. Regarding learning
Based on the proportion of individual versus group
activities in your course, which of the following best
represents the PREDOMINANT learning mode in your
course:
a. While there may have been some group and or class
interactions, a student primarily learned individually.
b. The students predominantly learned by interacting
online with other course participants.
References
[1] M. Alavi, Computer-mediated collaborative learning: an empiri-
cal evaluation, MIS Quarterly 18, 1994, pp. 159–174.
[2] M. Alavi, D.E. Leidner, Research commentary: technology-
mediated learning—a call for greater depth and breadth of
research, Information Systems Research 12(1), 2001, pp. 1–10.
[3] M. Alavi, D.E. Leidner, Review: knowledge management and
knowledge management systems: conceptual foundations and
research issues, MIS Quarterly 25(1), 2001, pp. 107–136.
[4] M. Alavi, G. Marakas, Y. Yoo, A comparative study of dis-
tributed learning environments on learning outcomes, Informa-
tion Systems Research 13(4), 2002, pp. 404–416.
[5] M. Alavi, B.C. Wheeler, J.S. Valacich, Using IT to re-engineer
business education: an exploratory investigation of collaborative
telelearning, MIS Quarterly 19, 1995, pp. 293–312.
[6] J.B. Arbaugh, Managing the on-line classroom: a study of
technological and behavioral characteristics of web-based
MBA courses, Journal of High Technology Management
Research 13, 2002, pp. 203–223.
[7] J.B. Arbaugh, How much does ‘‘subject matter’’ matter? a study
of disciplinary effects in on-line MBA courses Academy of
Management Learning & Education 4, 2005, pp. 57–73.
[8] J.B. Arbaugh, Is there an optimal design for on-line MBA
courses? Academy of Management Learning & Education 4,
2005, pp. 135–149.
[9] J.B. Arbaugh, R. Benbunan-Fich, In defense of quantitative
methods to research networked management learning: a reply
to Hodgson and Watland, Management Learning 35, 2004, pp.
117–124.
[10] R. Benbunan-Fich, Improving education and training with infor-
mation technology, Communications of the ACM 45(6), 2002,
pp. 94–99.
[11] R. Benbunan-Fich, S.R. Hiltz, Mediators of the effectiveness of
online courses, IEEE Transactions on Professional Communica-
tion 46(4), 2003, pp. 298–312.
[12] J.M. Carroll, R.L. Mack, C.H. Lewis, N.L. Grischkowsky, S.R.
Robertson, Exploring exploring a word processor, Human–Com-
puter Interaction 1, 1985, pp. 283–307.
 R. Benbunan-Fich, J.B. Arbaugh / Information & Management 43 (2006) 778–793
[13] J.F. Chizmar, M.S. Walbert, Web-based learning environments
guided by principles of good teaching practice, Journal of
Economic Education 30(3), 1999, pp. 248–264.
[14] N.W. Coppola, S.R. Hiltz, N.G. Rotter, Becoming a virtual
professor: pedagogical roles and asynchronous learning net-
works, Journal of Management Information Systems 18(4),
2002, pp. 169–189.
[15] S.A. Davis, R.P. Bostrom, Training end users: an experimental
investigation of the roles of the computer interface and training
methods, MIS Quarterly 17(1), 1993, pp. 61–81.
[16] S.A. Davis, F.D. Davis, The effect of training techniques and
personal characteristics on training end users of Information
Systems, Journal of Management Information Systems 7, 1990,
pp. 93–110.
[17] A.R. Dennis, B.H. Wixom, R.J. Vandenberg, Understanding fit
and appropriation effects in Group Support Systems via meta-
analysis, MIS Quarterly 25(2), 2001, pp. 167–193.
[18] M. Gallivan, R. Benbunan-Fich, A framework for analyzing
levels of analysis issues in studies of E-collaboration, IEEE
Transactions on Professional Communication 48(1), 2005
March, pp. 87–104.
[19] J.K. Gallini, A framework for the design of research in technol-
ogy-mediated learning environments: a sociocultural perspec-
tive, Educational Technology 41(2), 2001, pp. 15–21.
[20] V. Grover, T. Davenport, General perspectives on knowledge
management: fostering a research agenda, Journal of Manage-
ment Information Systems 18(1), 2001, pp. 5–23.
[21] L. Harasim, On-line education: an environment for collaboration
and intellectual amplification, in: L. Harasim (Ed.), On-line
Education: Perspectives on a New Environment, Praeger, New
York, 1990, pp. 133–169.
[22] S.R. Hiltz, The Virtual Classroom: Learning without Limits Via
Computer Networks, Ablex Publishing Corporation, Norwood,
NJ, 1994.
[23] S.R. Hiltz, M. Turoff, What makes learning networks effective?
Communications of the ACM 45(4), 2002, pp. 56–59.
[24] W.L.D. Hung, D.-T. Chen, Technologies for implementing social
constructive approaches in instructional settings, Journal of
Technology and Teacher Education 7(3), 1999, pp. 235–256.
[25] D. Jonassen, Thinking technology, Instructional Technology
1993, pp. 35–37.
[26] R.C.W. Kwok, M. Khalifa, Effects of GSS on knowledge
acquisition, Information & Management 34(6), 1998, pp.
307–315.
[27] R.C.W. Kwok, J. Ma, D.R. Vogel, Effects of Group Support
Systems and content facilitation on knowledge acquisition,
Journal of Management Information Systems 19(3), 2002, pp.
185–229.
[28] D.E. Leidner, M. Fuller, Improving student learning of concep-
tual information: GSS-supported collaborative learning vs. indi-
vidual constructive learning, Decision Support Systems 20,
1997, pp. 149–163.
[29] D.E. Leidner, S.L. Jarvenpaa, The use of information technology
to enhance management school education: a theoretical view,
MIS Quarterly 19, 1995, pp. 265–291.
[30] S.-S. Liaw, Designing the hypermedia-based learning environ-
ment, International Journal of Instructional Media 28, 2001, pp.
43–56.
[31] K.H. Lim, L.M. Ward, I. Benbasat, An empirical investigation of
computer system learning: comparison of co-discovery and self-
discovery methods, Information Systems Research 8(3), 1997,
pp. 254–273.
793
[32] J. Lu, C.-S. Yu, C. Liu, Learning style, learning patterns and
learning performance in a WebCT-based MIS course, Informa-
tion & Management 40(6), 2003, pp. 497–507.
[33] J. Nunnally, Psychometric Theory, McGraw Hill, New York,
NY, 1978.
[34] G. Piccoli, R. Ahmad, B. Ives, Web-based virtual learning
environments: a research framework and a preliminary assess-
ment of effectiveness in basic IT skills training, MIS Quarterly
25, 2001, pp. 401–426.
[35] S. Raudenbusch, A. Bryk, Hierarchical Linear Models: Applica-
tions and Data Analysis Methods, Sage Publications, Thousand
Oaks, CA, 2002.
[36] A.P. Rovai, Sense of community, perceived cognitive learning,
and persistence in asynchronous learning networks, The Internet
and Higher Education 5, 2002, pp. 319–332.
[37] M. Rungtusanatham, L.M. Ellram, S.P. Siferd, S. Salik, Toward a
typology of business education in the internet age, Decision
Sciences Journal of Innovative Education 2, 2004, pp. 101–
120.
[38] A.C. Sherry, C.P. Fulford, S. Zhang, Assessing distance learners’
satisfaction with instruction: a quantitative and a qualitative
measure, The American Journal of Distance Education 12(3),
1998, pp. 4–28.
[39] S.J. Simon, J.M. Werner, Computer training through behavior
modeling, self-paced, and instructional approaches: a field
experiment, Journal of Applied Psychology 81(6), 1996, pp.
648–659.
[40] R. Walczuch, R. Watson, Analyzing group data in MIS research:
including the effect of the group, Group Decision and Negotia-
tion 10(1), 2001, pp. 83–94.
[41] P. Webb, Student interaction and learning in small groups,
Review of Educational Research 52, 1982, pp. 421–445.
[42] J. Webster, P. Hackley, Teaching effectiveness in technology-
mediated distance learning, Academy of Management Journal
40, 1997, pp. 1282–1309.
Raquel Benbunan-Fich is an associate professor at the SCIS Depart-
ment in the Zicklin School of Business, Baruch College, City Uni-
versity of New York. She received her PhD in Management
Information Systems from Rutgers University, Graduate School of
Management. Her research interests include educational applications
of computer-mediated communication systems, asynchronous learn-
ing networks, evaluation of Web-based systems and e-commerce. She
has published articles on related topics in Communications of the
ACM, Decision Support Systems, Group Decision and Negotiation,
IEEE Transactions on Professional Communication, Information &
Management, International Journal of Electronic Commerce, Journal
of Computer Information Systems and other journals.
J.B. Arbaugh is the Curwood Endowed professor and a professor of
Strategy and Project Management at the University of Wisconsin
Oshkosh. He is an associate editor of Academy of Management
Learning & Education. Ben’s research interests are in online manage-
ment education, international entrepreneurship, the management of
rapidly growing firms, and the intersection between spirituality and
strategic management research. Some of his recent publications
include articles in Academy of Management Learning & Education,
Decision Sciences Journal of Innovative Education, Management
Learning, the Journal of Management, Spirituality, and Religion,
the Journal of Enterprising Culture, and the Journal of Management
Education.