The Evaluation of SHU3DED Cyber Campus – A pilot study
Louis Nisiotis, Martin Beer, Elizabeth Uruchurtu
Sheffield Hallam University
Cultural Communication and Computing Research Institute (C3RI)
Sheffield, United Kingdom
louis.nisiotis@students.shu.ac.uk, m.beer@shu.ac.uk, e.uruchurtu@shu.ac.uk
Abstract— This paper presents the preliminary findings of an
empirical study that aims to evaluate the efficacy of 3D virtual
worlds for synchronous distant learning activities. A pilot
study was conducted to evaluate the prototype developed, the
procedures and instruments of data collection, to prepare the
study investigating whether virtual worlds can help students
participate more effectively in learning activities. The
theoretical framework is presented, together with the proposed
prototype, method and results of the pilot study, its associated
limitations and the future work that we intend to perform.
Keywords: virtual worlds; cyber campus; MUVE; pilot study;
I. INTRODUCTION
The use of technology for education has drawn a lot of
attention in the past decade, concentrating on how to
enhance learning activities and support students’ needs.
However, some students face issues that restrict them from
attending the university, missing important learning
experiences through personal interaction with both the staff
and their peers. One of the solutions used to facilitate
computer-mediated learning is virtual worlds, in the form of
cyber campuses. The ability of cyber campuses to allow
students participate more effectively in learning activities
has been empirically investigated, using a prototype that has
been developed. To prepare for this investigation, a pilot
study has been conducted to assess the procedures and data
collection instruments and is discussed in this paper.
II. BACKGROUND AND CONTEXT
There are many reasons why some students cannot
attend university. To support them, a range of learning
material delivery tools e.g. Blackboard are provided,
facilitating E-Learning technologies through distance. E-
Learning is capable of supporting learning using various
multimedia techniques over the Internet, enabling users to
exchange knowledge using asynchronous or synchronous
communications in their own time and pace [1], providing
opportunities for more accessible education. However, the
conventional distant learning environments currently in use
are considered “socially dry” [2], therefore the important
learning experiences that can be obtained when students
attend and participate in learning cannot be replicated. As a
result of technology mediated learning through distance,
traditional E-Learning lacks of human-to-human interaction
[3]. This has been taken into consideration and the use of
virtual worlds in the form of cyber campuses has started to
develop. Cyber campuses are specially designed meeting
points that operate on Multi-User Virtual Environments
(MUVE’s), in which users gather and exchange learning
materials, communicate and collaborate in a state of the art
3D environment [4]. Cyber campuses offer shared 3D
spaces that support variety of multimedia presentation
techniques, tools for synchronous interaction and
communication, enhancing socialization among users [5].
Using a virtual representations of themselves (avatars),
students navigate and interact with their peers and learning
materials in the virtual world and can also create and
interact with virtual objects, achieving the “immersion”
feeling of being there [6]. Taking these attributes into
account, cyber campuses are considered as an effective
vehicle for learning support [7].
III. THEORETICAL FRAMEWORK
The theoretical framework of this research is based on 1)
the methodology for evaluating the efficacy of MUVE’s for
synchronous distant learning proposed by [8] and 2) on
empirical investigation of the ability of virtual worlds to
overcome barriers of access and participation in learning. In
a MUVE, learning is strongly related to student perceptions
of presence, communication, awareness and the feeling of
belonging to a community [8]. This methodology enables us
to evaluate the efficacy of our cyber campus prototype to
support synchronous distant learning activities. Our research
then aims to investigate how virtual worlds may allow
students participate more effectively in learning activities.
One of the most important features of 3D virtual
environments is the strong presence sensation that they
provide compared to 2D environments [9], generating
immersion, which is the feeling of the user’s actual presence
in the environment [10]. Immersion enhances collaboration
and socialization [7] and is strongly related to learning;
“increasing presence also increases learning and
performance” [8]. According to [8], presence factor is not
enough when measuring users experience on MUVE and
they propose additional factors comprising awareness,
communication and sociability. Awareness relates to the
ability to feel the existence of other users in the
environment, communication concerns the non verbal
communication that complements verbal and sociability
relates to the use of a social space that facilitates
collaboration, socialization, access to information and the
sense of belonging to a community [8].
 IV. THE DESIGN OF SHU3DED 3) Instruments
To perform this investigation, the SHU3DED (Sheffield To collect empirical data from this pilot study, students
Hallam University 3D EDucation) cyber campus has been completed a post experiment survey at the end of each
developed using Opensim server (www.opensimulator.org). session. This survey has been adapted from [9], [12] and [8].
To provide learning functionalities to the prototype, the We have also recorded the chat communication between
Moodle learning management system (www.moodle.com) students during sessions for further analysis.
together with Sloodle (www.sloodle.org) were deployed. VI. RESULTS
Moodle can be partially integrated into virtual worlds using
Sloodle, offering a series of learning functionalities to be Due to the fact that sessions were conducted under the
ported. Sloodle then establish communication with Moodle same experimental procedures, results were first aggregated
through objects in the virtual world, allowing the design of and analyzed and then examined individually. The presence
activities to be retrieved within the environment. Details of results were analyzed first (Table I). The total presence
design influences, layout and features of the prototype are score is calculated by aggregating all presence items
discussed in [11]. (Max=133, Min=19). It can be observed that students
expressed high perceptions of presence in the virtual world
V. THE PILOT EVALUATION (101.6), even if they were all in the same physical setting.
The purpose of this pilot study is to evaluate the efficacy Individual factors analysis revealed that users had good
of the prototype to support synchronous learning activities. sense of control (CF) in the environment (M=5.3, SD=0.6).
Procedures and data collection instruments have also been Similar results have been obtained for SF (M=5.4, SD=0.8),
evaluated to identify issues that might influence the validity RF (M=5.1, SD=0.8) and DF (M=5.5, SD=0.7), with similar
and reliability of the intended study. This pilot study aims to data dispersion. The additional factors and environment
replicate the setting and procedures to be used as far as evaluation results are summarized in table II. Students
possible, to ensure good operation in practical environment, expressed high sociability, awareness and communication
conditions management and procedures simulation. perceptions, even if they used only the textual chat and
1) Participants gestures to communicate. Results corresponding to the
This pilot study was conducted in three sessions specific evaluation for the virtual world, productivity and
involving three different groups. 23 undergraduate general satisfaction also revealed high mean (M=4.2) with
computing students of Sheffield Hallam University were relatively low data dispersion. Further investigation has
recruited comprising three groups of 7, 13 and 3 students been performed to identify differences between groups,
(21 of males and 2 females, between 19 and 21 years old). revealing no significant differences; therefore we accepted
2) Procedures the aggregated data analysis.
TABLE I.
Three 60-minute sessions were performed in a university
computer room. Students logged in the system and spend Presence Questionnaire Results
Total Presence Presence Factors
few minutes in the orientation area to learn its basic
functionality. Students then “teleported” to the lecture room, Total Scaled CF SF RF DF
where the principal author delivered an oral presentation Mean 101.6 5.3 5.3 5.4 5.1 5.5
based on the educational use of virtual worlds (Fig. 1a). SD 11.6 .6 .6 .8 .8 .7
Since students and presenter were in the same room, the oral Max 118 6.2 6.4 6.8 6.5 6.7
aspect of the presentation took place in real world and Min 76 4 4 3.3 3.5 4
presentation slides appeared in the virtual world, thus Legend: CF=Control Factors, SF=Sensory Factors, RF=Realism Factors,
students were looking at their screen while listening to the DF=Distraction Factors.
presentation. Students then undertook a virtual quiz and then
teleported to the tutorial room where discussion based on TABLE II.
the use of virtual worlds in education took place (Fig. 1b). Additional Factors Results
Efficacy Factors Environment Evaluation
SOC COM AW PRO SAT CVE
Mean 3.9 4.2 4.3 4.1 4.4 4.3
SD .2 .3 .4 .6 .5 .3
Max 4.3 4.7 5 5 5 5
Min 3.6 3.7 3.7 2 4 3.9
Legend: SOC=Sociability, COM=Communication, AW=Awareness, PRO
=Productivity, SAT=Satisfaction, CVE=Collaborative Virtual Environment

Figure 1 – Examples of activities during the pilot study

 VII. DISCUSSION
The results of this evaluation reveal that students have
positively evaluated the prototype. During the experience,
students achieved high sense of presence, reporting good
sense of control, high sensory and perceived realism of the
environment with minimum distractions. Presence results
denote that students have immersed in the virtual world
during the experience, achieving the feeling of “being
there”. The students’ perceptions of being in a virtual
setting together with their peers have been also evaluated
positively, with positive perceptions towards the ability of
the virtual world to support sociability. Students reported
that they were aware of others around them, what they were
doing and what was going on in the virtual world. Also
effective communication and interaction among them has
been reported. Results suggest that the prototype supports
effective communication and social interaction. Students
have also positively evaluated the design of the prototype,
considering the design comfortable, allowing them to be
productive and they were generally satisfied from the
experience. Results have also been confirmed after
analyzing the chat logs of the communications as
established among students, revealing positive opinions:
“Panos Creed: Its far more engaging than slideshows or
just chatting on forums… highly expandable too”.
“Larry Cosmos: It would be useful for teamwork where
some wouldn't be able to attend in person”.
“Tom Hallam: It has possibilities for enabling teamwork
when you are at home and make lectures more interesting”.
During this pilot study, we have monitored the system
behavior, evaluated the scenario procedures and the data
collection instruments. During the sessions, the system was
behaving properly i.e. no crashes, lag, graphics rendered
correctly. The procedures are considered appropriate to
demonstrate the learning potentials of the prototype and the
instruments capable to provide reliable empirical data.
VIII. LIMITATIONS
The most important limitation of this pilot study relates
to the fact that the experiments took place in controlled
environment, in which students were located in the same
physical room and presentations took place orally in the
physical classroom rather than within the virtual world. The
concept of our research is to evaluate the perceptions of
students in distance and this have most likely affected the
users perceptions of presence. Also, due to the time
limitation of the sessions, students did not communicate and
interact long enough to develop working relationships and
also the procedures were shortened. This could have
affected communication, awareness and sociability results.
In addition, the three sessions were conducted using
different numbers of students in each group and this may
have also affected the results. We could not investigate this
further in this study and we aim to look at this further in
future.
IX. CONLCUSION AND FUTURE WORK
This paper presents a proposed cyber campus prototype
and the preliminary findings of a pilot study evaluating its
efficacy to support learning activities, the procedures and
instruments of data collection. Results obtained so far
suggest that the environment is stable, procedures are
capable enough to demonstrate the prototype ability to
support synchronous learning activities and the instruments
of data collection are valid and reliable. Further work to
investigate how virtual worlds can help students participate
more effectively in learning activities is under way.
X. ACKNOWLEDGEMENT
Research partially funded by A.G Leventis Foundation.
XI. REFERENCES
[1] R. W. Lau, N. Y. Yen, F. Li, and B. Wah, "Recent
Development in Multimedia E-Learning Technologies," World
Wide Web, pp. 1-10, 2013.
[2] D. Carr, M. Oliver, and A. Burn, "Learning, Teaching
and Ambiguity in Virtual Worlds," presented at the Relive08
Proceedings of Researching Learning in Virtual Environments,
2008.
[3] R. Sanders, S. Bronack, A. Cheney, J. Tashner, D. Riedl,
and R. Gilman, "Education in the Zone: Dynamic Learning
Communities in a 3d Virtual World," presented at the IADIS
International Conference on Web Based Communities, 2007.
[4] E. Prasolova, A. Sourin, and O. Sourina,
"Cybercampouses: Design Issues and Future Directions," Visual
Computer Journal, 2006.
[5] M. Chau, A. Wong, M. Wang, S. Lai, K. W. Y. Chan, T.
M. H. Li, D. Chu, I. K. W. Chan, and W.-k. Sung, "Using 3d
Virtual Environments to Facilitate Students in Constructivist
Learning," Decision Support Systems, 2013.
[6] P. Cronin, "An Exploratory Case Study in the Use of
Multi-User Virtual Environments to Support and Enhance a
Community of Practice," Master of Science in Technology &
Learning, University of Dublin, 2011.
[7] D. Livingstone, J. Kemp, and E. Edgar, "From Multi-
User Virtual Environment to 3d Virtual Learning Environment,"
Alt-J, vol. 6, pp. 139-150, 2008.
[8] A. De Lucia, R. Francese, I. Passero, and G. Tortora,
"Development and Evaluation of a Virtual Campus on Second
Life: The Case of Seconddmi," Computers & Education, vol. 52,
pp. 220-233, 2009.
[9] B. Witmer and M. Singer, "Measuring Presence in
Virtual Environments: A Presence Questionnaire," Presence, vol.
7, pp. 225-240, 1998.
[10] J. Seng and E. Edirisinghe, "Teaching Computer Science
Using Second Life as a Learning Environment," presented at the
Proceedings ascilite Singapore, 2007.
[11] L. Nisiotis, M. Beer, and E. Uruchurtu, "The Shu3ded
Cyber Campus Prototype," IEEE Learning Technology Newsletter,
vol. 13, pp. 14-17, 2011.
[12] K. Kreijns, P. A. Kirschner, W. Jochems, and H. Van
Buuren, "Measuring Perceived Sociability of Computer-Supported
Collaborative Learning Environments," Computers & Education,
vol. 49, pp. 176-192, 2007.