Sweet Final Report FINAL

JISC Final Report
Project Information
Project Identifier
sweet
Project Title
Synchronous Web Enabled Employee Training
Project Hashtag
Start Date
21 January 2011
End Date
31 December 2011
Lead Institution
Jewel & Esk College
Project Director
Wendy MacAdie, Head of Centre: Community Engagement
Project Manager
Donald Steele
Contact email
WMacAdie@jec.ac.uk
Partner Institutions
Project Web URL
ELVON (East Lothian Voluntary Organisations Network)

REHIS (Royal Environmental Health Institute of Scotland)
Scottish Seabird Centre
Soroba Training CIC
http://mycourses.jec.ac.uk/wordpress/sweet
Programme Name
JISC Learning and Teaching Innovation Grants: SWaNI FE (10/10)
Programme Manager
Rob Englebright
Document Information
Author(s)
Donald Steele
Project Role(s)
Project Manager
Date
13 January 2012
URL
Access
This report is for general dissemination
Filename
sweet_final_report_FINAL.doc
Document History
Version
Date
Comments
13/01/2012
Submission date extended in agreement with JISC Programme

Manager.
Table of Contents
1
ACKNOWLEDGEMENTS..............................................................................3
PROJECT SUMMARY..................................................................................4
MAIN BODY OF REPORT.............................................................................5

3.1
3.2
3.3
3.4
3.5
PROJECT OUTPUTS AND OUTCOMES...............................................................................5

HOW DID YOU GO ABOUT ACHIEVING YOUR OUTPUTS / OUTCOMES?......................................6
WHAT DID YOU LEARN?............................................................................................. 13
IMMEDIATE IMPACT.................................................................................................... 15
FUTURE IMPACT....................................................................................................... 17
CONCLUSIONS........................................................................................18
RECOMMENDATIONS...............................................................................19
IMPLICATIONS FOR THE FUTURE..............................................................21
REFERENCES..........................................................................................22
APPENDICES ..........................................................................................22
APPENDIX A:
GLOSSARY OF ACRONYMS USED IN REPORT
. 23
APPENDIX B:
COMPLETED BBB USER ACCEPTANCE TEST REPORT
..24
APPENDIX C:
BIGBLUEBUTTON: RECOMMENDED COMPUTER SETUP FOR
OFF-CAMPUS USERS ..26
APPENDIX D:
REHIS PILOT COURSE EVALUATION REPORT
28
APPENDIX E:
REHIS STUDENT COURSE EVALUATION SUMMARY REPORT
.43
APPENDIX F:
SYNCHRONOUS COLLABORATIVE LEARNING CASE STUDY
50
APPENDIX G:
SYNCHRONOUS COLLABORATIVE LEARNING: COURSE
PLANNING, SETUP AND DELIVERY
MODEL
..54
APPENDIX H:
SYNCHRONOUS COLLABORATIVE LEARNING / BIG BLUE
BUTTON TRAINING
SPECIFICATION: COURSE TUTOR COMPETENCES
68
1 Acknowledgements
The SWEET Project Team would like to thank the following people and organisations for their valuable
and valued - financial, professional, technical and personal support without which our work and
achievements would have been much diminished:
JISC
JISC Learning and Teaching Innovation Grants: SWaNI FE (10/10) for funding the SWEET Project
Rob Englebright, Programme Manager: JISC e-Learning Programme for encouragement, feedback
and support throughout the Project.
BigBlueButton
Fred Dixon, CEO, Blindside Networks Inc, Ottawa, Ontario for technical and professional support of
BigBlueButton
Evaluation
Andrew Comrie, Project Director, ELRAH (Edinburgh, Lothians, Fife & Borders Regional Articulation
Hub) for professional guidance and support with project evaluation.
SWEET Project Steering Group members
For personal and professional guidance throughout the Project:
James Bruce, Chief Executive, ELVON, also for arranging student participation in the pilot course.
Ian Crawford, Trustee, Soroba Training PLC, also for arranging student participation in the pilot
course.
Graham Walker, Director of Training, REHIS, also for encouragement in developing an e-assessment
methodology and support of a formal proposal to REHIS
Scottish Seabird Centre, also for arranging student participation in the pilot course.
Anne Donohue, Head of Centre: Tourism, Jewel & Esk College
Wendy MacAdie, Head of Centre: Community Engagement, Jewel & Esk College
Jackie Doody, Head of Faculty: Creative Industries & Art, Jewel & Esk College
2 Project Summary
Traditional college delivery of employee training, particularly to SMEs, is limited by distance from
workplace to campus. Open learning requires employees to learn in relative isolation while
workbased learning requires sufficient employees to justify the additional college expenses.
Synchronous collaborative learning (SCL) offers an affordable solution for employers and colleges by
enabling work-based learners to join live campus-based classes via web-conferencing. The SWEET
Project developed and tested a course delivery model using the open source BigBlueButton webconferencing system and Moodle to deliver the REHIS Elementary Food Hygiene course
simultaneously to on- and off-campus learners. The methodology is transferable across the further
education curriculum.
Open source ICT applications are affordable to further and higher education as they incur no
purchase or licencing costs. The project established that the Jewel & Esk College ICT and VLE
technical support staff successfully integrated the open source BigBlueButton web-conferencing
system into its open source Moodle virtual learning environment and completed technical testing
using 60 hours of staff time. The REHIS course tutors worked with a team of College Learning
Technologists to design, deliver and evaluate a model for delivering the REHIS course simultaneously
to classroom-based and off-campus online student cohorts.
The Project established that simultaneous delivery of training courses to workbased and on-campus
student cohorts can be made to work pedagogically. 24 of the 25 students who completed the final
course examination passed. For SCL to work technically, web-enabled conferencing requires local
computer workstations to meet a specified minimum software, connectivity and audiovisual hardware
requirement. Technical difficulties with remote user connectivity, however, is a current general issue
for the further and higher education community which it must tackle in managing and delivering live
web-conference courses. Tutor technical competence and confidence in using web-enabled
conferencing backed up with adequate expert, non-expert and peer support in college is essential for
the SCL model to work.
The Project provides colleges with a documented standard SCL model, comprehensive evaluation
report, practical case study, tutor training specification, BigBlueButton user guides, employer
requirements checklist, student evaluation process and remote candidate e-assessment methodology,
all available on the SWEET Project website - http://mycourses.jec.ac.uk/wordpress/sweet.
3 Main Body of Report

3.1 Project Outputs and Outcomes
Output / Outcome Type
(e.g. report, publication,
software, knowledge built)
Brief Description and URLs (where applicable)

For documents flagged as available on the project website go to:
BigBlueButton acceptance test

criteria and template
A user acceptance test template was developed and applied when

BigBlueButton was handed over to the system users following
installation. Appendix B provides the criteria applied in a template
format. Blank template available on project website.
Integrated open source

synchronous collaborative
learning system built and tested
from BigBlueButton and Moodle;
A fully operational Moodle/BigBlueButton system has been in service

at Jewel & Esk College since April 2011. The system has been tested
by delivery of the REHIS Elementary Food Hygiene course
simultaneously to classroom-based cohorts and remote students
participating in the class via BigBlueButton, with Moodle used to
support elements of course delivery.
Technical report and specification

for integrating BigBlueButton with
Moodle
SCL case study
A filmed report by the Jewel & Esk College Systems Manager has
been recorded. http://vimeo.com/user8642960/sweet
SCL tutor comptences

specification
A tutor competences specification aimed at BigBlueButton system

users and covering SCL delivery methods was developed. Available
on project website.
Integrated synchronous
collaborative learning system
user guide for tutors.
A tutor guide to BigBlueButton, covering both technical system usage

and course development/delivery guidance, has been prepared.
Available on project website.
Integrated synchronous
collaborative learning system
user guide for students
A student guide to BigBlueButton, covering both technical system

usage and preparation for participating in online course sessions, has
been prepared. Available on project website.
Test report on REHIS

learning delivery model with
college-based student cohort
A detailed evaluation of the course delivery model and experience has

been produced, drawn from feedback provided by course tutors,
Learning Technologists and students, with proposed solutions to
issues raised. Available on project website.
Package of course resources

designed to support delivery of
REHIS course by synchronous
collaborative learning
A revised presentation, redesigned for the SCL delivery methodology

and supported by a specially commissioned video has been produced.
Due to their commercial sensitivity, these resources cannot be shared
with the wider FE community.
Employer requirements checklist
A guide and checklist of employer requirements for supporting

employee training via SCL has been produced. Available on project
website.
A case study (Synchronous Collaborative Learning case study 1:

REHIS Elementary Food Hygiene at Jewel & Esk College) describes
the experience of running the pilot course, outlining course delivery
arrangements; changes relating to tutor presentation, timing and
access to online resources; and evaluates the student experience.
Available on project website.
Output / Outcome Type

(e.g. report, publication,
software, knowledge built)
Brief Description and URLs (where applicable)

For documents flagged as available on the project website go to:
REHIS pilot SCL cohorts trained
18 classroom-based and 8 online students completed the REHIS

Elementary Food Hygiene over 8 separate class sessions delivered
using the SCL methodology.
Student evaluation process
A student evaluation process was developed which included an

online SCL student evaluation questionnaire and a student focus
group. The questionnaire, focus group specification and student
evaluation report are available on the project website.
Centre of expertise on
BigBlueButton implementation
project
The SWEET Project methodology provides a template for other

Colleges to use when planning/implementing the incorporation of
BigBlueButton or equivalent system into their VLE. Colleges and
HEIs may approach Jewel Esk College for technical, pedagogical and
project implementation advice gained from the SWEET Project.
Learning, support and

assessment model developed for
learning.
A Synchronous Collaborative Learning: course planning, setup and

delivery model has been developed which draws from the experience
of the SWEET Project participants but is presented as a generic
delivery model which can be adopted by any educational institution
and applied to any course. Available on project website.
Remote candidate e-assessment

methodology
A detailed methodology for managing formal assessment at remote

locations, with local invigilation supported by online invigilation using
BBB has been developed for review by REHIS. The methodology can
be adapted to other formal assessment requirements laid down by
other awarding bodies. Available on project website.
3.2 How did you go about achieving your outputs / outcomes?

Aims and objectives
Broadly the SWEET Project aimed to pilot and evaluate the delivery of work-based training using
synchronous collaborative learning, supported by BigBlueButton and Moodle. In order to test and
evaluate the methodology within a specific operational context the project aimed to deliver the REHIS
Elementary Food Hygiene qualification directly to employees in the Hospitality and Tourism sector but
simultaneously in the classroom and in the workplace via BigBlueButton and Moodle.
The projects objectives were to:
1. Establish, pilot and evaluate a technology-enhanced learning methodology which enables
partner employers to train staff directly in the workplace while simultaneously communicating
directly with tutors and other students at a distance.
2. Develop and test a working methodology capable of supporting similar work-based learning
initiatives across both the FE and Hospitality and Tourism sectors.
3. Address problems of learning in isolation from lecturer and peers using a target group of
learners who themselves present particular challenges to designers of the learning
programme in terms of learner engagement and sustaining motivation.
4. Establish and evaluate the costs/benefits of applying the methodology in order to address
traditional barriers faced by employers, particularly SMEs, in developing their workforce,
namely:
6
the costs of releasing staff;

travel time; and
additional charges by training providers where cohorts are small.
5. Scope the integration of BigBlueButton with Moodle, integrate the systems, test the
functionality of the enhanced overall system and provide a record of the process.
6. Produce a proven technical process with setup instructions for other Moodle users who wish
to integrate BigBlueButton to their VLE, linking this to existing work elsewhere and providing a
specific FE and workbased learning context to system deployment and use.
Project methodology
The Project plan allowed for all activities and deliverables to be completed within a single calendar
year, running from January-December 2011. The project management and delivery structure
comprised:
Project partnership, comprising:
Lead institution - Jewel & Esk College accountable for delivery of the project aims and
objectives within the allocated budget.
Employers East Lothian Voluntary Organisations Network (ELVON); Soroba Training PLC,
Oban; Scottish Seabird Centre, North Berwick.
Awarding body Royal Environmental Health Institute for Scotland (REHIS), responsible for
ensuring that the centres approved to delivery and formally assess their courses meet the
required standards.
Project management structure, comprising:
Project Director, directly responsible at College SMT level for directing and monitoring the
Project Manager and holding final authorisation to deploy the project budget.
Project Steering Group, comprising senior representatives from the lead institution, Project
partners and chaired by the Project Director with the Project Manager in attendance.
Project Manager, responsible for the management of operational activities leading to the
delivery of the project plan.
Project Team, comprising the key lead institution staff responsible for technical system setup
and maintenance, course planning and delivery, tutor/student support in course delivery via
Moodle and BigBlueButton and project administration.
The Project Steering Group which met 3 times in June, October and December in order to:
Receive and approve the Project Managers progress report towards completion of
workpackages;
Review the project risk assessment;
Review and approve deployment and expenditure of the project budget;
Agree actions required to address specific issues arising.
Directly engaging the Project partners as Steering Group members ensured their direct and continued
involvement in management decision making and, when required, direct action in taking forward
operational issues within their own institutions. Employer representatives were instrumental in
ensuring the delivery of student cohorts for the pilot training course while the awarding body
representative championed the adoption of the projects e-assessment model within REHIS.
Project implementation phases

The SWEET Project, broken down and summarised as a series of key implementation phases,
adopted a standard approach familiar in most projects established to introduce, deploy, evaluate and
mainstream a new IT system.
Phase 1: System procurement and technical setup (Workpackage 2)
The Colleges ICT Systems Manager and Virtual Learning Support & Development Technologist
worked together through the system procurement and setup phase. The Systems Manager procured
the hardware and software required to establish a fully operational networked BigBlueButton system
and the Virtual Learning Support & Development Technologist integrated the system with the
Colleges Moodle VLE.
The BigBlueButton website provides documentation and comprehensive technical guidance required
to install the application. Both the ICT Systems Manager and Virtual Learning Support &
Development Technologist reported that following the installation instructions provided on the BBB
website enabled them to install, configure and test the Jewel & Esk College system without
encountering any significant technical issues.
The procurement, installation and testing of the hardware/software took place over February-April
2011 as follows:
Task
By
Time taken
Install, configure & test

BigBlueButton/Moodle plug-in on test
server
Virtual Learning Support & Development

Technologist (Feb)
10 hours
Rack & stack server; run hardware tests
ICT Systems Manager (Mar)
8 hours
Install & test operating system & BBB

server
6 hours
Install, configure & test BBB/Moodle

plug-in on production server

Technologist (Mar)
4 hours
Research, install & tests BBB server
10 hours
Install, configure & test BBB server

Technologist (Mar)
10 hours
Configure & test BBB server & Moodle

plug-in

Technologist (Apr)
4 hours
Acceptance test BBB server & Moodle

plug-in

Technologist (Apr)
6 hours
Assuming access to the in-house technical expertise required to install, configure and test
BigBlueButton and integrate it with an existing Moodle VLE, a College (or any other educational
institution) considering developing web-conferencing functionality should therefore allow for an overall
investment of some 60 hours (10 days) technical support staff time to set up the system and make it
ready for handover to academic staff and/or academic support services.
A recorded interview with the ICT Systems Manager, available on the SWEET Project website,
provides an overview and very positive evaluation of the technical setup process system functionality.
Phase 2: User orientation and testing (Workpackage 3)
A key criterion of any systems potential must be the ease with which its typical user base can learn to
use and apply it within a real operating environment. The original project plan provided for a pilot
course development and delivery team comprising 2 experienced members of academic staff with
extensive experience of delivering the REHIS Elementary Food Hygiene course in both the College
8
and the workplace. While both tutors were users of standard IT applications, neither were expert IT
users and neither had experience of using web-conferencing tools within course delivery. The tutors
therefore satisfied the criterion of being typical users, making them ideal testers through which to
evaluate the viability of introducing and rolling out the application within the College.
Jewel & Esk College maintains a team of 8 Learning Technologists who are responsible for supporting
classroom delivery and, crucially in the context of the SWEET project, for supporting the development,
maintenance and delivery of Moodle courses. Given the need to ensure that the outcomes of the
project would be, as far as possible, sustainable for the College with an immediate transfer from pilot
to mainstream activity it was agreed early in Phase 1 to bring the Learning Technologists into the
project. This not only provided essential support for the pilot tutors and students but also placed BBB
academic support within the most appropriate College operational service.
The original project plan over-estimated the deployment of technical support staff hours required to
install and set up the integrated BigBlueButton/Moodle system. Conversely, the estimated period of 2
months within which to develop the tutors and Learning Technologists into confident BigBlueButton
users proved a considerable under-estimate.
Individually, the standard functions provided by BBB are technically straightforward to operate,
comprising:
Web participants students and tutors may join a class session simultaneously from an
internet-connected computer located (in principle) anywhere in the world.
Voice participants online participants use a headphone/microphone headset to

communicate by voice, with the tutor moderating voice question/discussion sessions.
Web cam online participants also use a web camera.
Group and private chat students and tutors may communicate by keyboard in group and
one to one chat/discussion sessions.
Presentation tutors use a built-in electronic whiteboard to display Powerpoint presentations

as part of the course delivery.
Desktop sharing tutors may display all or part of their computer desktop, enabling them to
show files and resources (eg film recordings) which may require access to applications
accessible only on their local computer.
The BBB website provides short video demonstrations of system functions for students and tutors. As
the system was designed to be simple enough for users to learn its functions quickly, the tutors and
Learning Technologists considered a self-instructional approach to be the most appropriate method of
training and development.
As an initial orientation exercise the Learning Technologists developed a user acceptance test
template for BigBlueButton and lead a user evaluation with the tutors. The test comprised developing
a simple user function template with the functions/operational criteria and an area for testers to report
findings. The findings were summarised and subsequently used to address technical issues.
Appendix B provides an example of the test template completed by a Learning Technologist.
The evaluation highlighted the requirement for computers used by all users to meet a minimum
software specification (see Appendix C) a key issue for using BBB as a medium for delivering workbased learning where the College exercises no control over local PC configurations. Within the
College itself, individual workstations do not all conform to a common software standard and
consequently did not all meet the minimum specification. This required the Project Team to identify
specific PCs for use in testing and delivering courses for checking and upgrading by ICT Technicians.
Phase 3: Curriculum design and development (Workpackage 4)
When brought into the Project, the REHIS tutors believed that the REHIS Elementary Food Hygiene
course delivered simultaneously from the classroom to remote students would require little adjustment
of the traditional delivery model. The nature of the course appeared to make it ideally suited for
delivery through BBB, being short and traditionally delivered over a single day. Interestingly, feedback
received from students who eventually completed the course clearly indicates that they too consider
the methodology to be better suited to short course delivery than extended learning programmes.
As the tutors became familiar with the scope and functions of BBB and in the light of their experience
with the first student cohort, however, they realised that the face to face model required a more
fundamental review and re-design for synchronous collaborative learning.
The following curriculum design issues emerged and were addressed:
ISSUE
The levels of previous educational
experience and achievement among Food
and Hospitality workers is below average.
SOLUTION
Use students own experience and tutors
fund of stories to illustrate points, lighten the
overall learning experience engage students
as learners.
Although short, the course is intensive and

covers a mandatory syllabus which leaves
little room for variation.
Break course into 3 two-hour sessions

delivered separately over 3 consecutive
weeks.
Make course resources available via Moodle
for students to use between course sessions
as preparation for and reinforcement of group
learning sessions.
Develop a robust online invigilation
methodology, using the audio-visual link
between tutor and student within the
invigilation process.
The syllabus satisfies a legislated employee

training requirement and requires an
invigilated end of course examination.
The inclusion of students who joined the

class online introduced a different course
induction requirement to that provided for the
Develop separate online student induction

package, sent to students in advance. Bring
students online 30 minutes before start of
10
ISSUE
classroom cohort.
SOLUTION
course and take them through the online
induction.
Audio feedback experienced when online

participants relayed audio communication
through open speakers
The Powerpoint presentation provided by
REHIS for course tutors was designed for
traditional face to face delivery and was not
compatible with presentation font.
Required tutors to mute speakers and act as

a conduit between the classroom and online
students
.
Powerpoint presentation redesigned to meet
the expected needs of both groups of
learners. Powerpoint saved as PDF and
uploaded in this format.
College-based students receive a tour of the

College kitchens for a first-hand view of food
hygiene management.
Commission film of kitchen tour with

voiceover for presentation to combined class
simultaneously via Moodle.
Online students joined the course as

individuals, physically isolated from one
another and the classroom cohort.
Reduce group exercise work in favour of

open forum activities and directed questions.
Phase 4: Employer liaison (Workpackage 5)

The Project initially planned to establish a memorandum of agreement with partner employers, setting
out the roles and responsibilities of partners and signed as a formal project document. In the event,
employers were invited to become members of the Project Steering Group and the REHIS tutors
already enjoyed a sound working relationship with REHIS and two of the employer partners. The
relationship between partners and the Project was therefore sustained on this semi-formal basis.
The final recruitment of students took place over a period of two weeks leading up to the delivery of
each of the 3 pilot courses. The lines of communication between the Project Team and workplace
supervisors who were asked to identify employees for training did become stretched as a result of the
short notice provided and the need to communicate directly with constituent groups within the ELVON
partnership. It is unlikely that the existence of a formal signed agreement would have eased this
process as the communication issues arose more from the short timescale allowed for recruitment,
pending the resolution of system performance issues encountered by the tutors and Learning
Technologists.
As the system is rolled out post-project, we recommend that new course delivery be scheduled only
once required BigBlueButton functional parameters are fully tested followed by a period of sufficient
length to ensure an effective student recruitment process.
Phase 5: Delivery of pilot REHIS course (Workpackage 6)
The REHIS Elementary Food Hygiene course was delivered to 3 separate student cohorts, each
comprising a group of classroom-based students joined by 1, 5 and 2 remote learners via
BigBlueButton. A full evaluation of the pilot course is provided in Appendix D with a summary of the
student evaluation provided in Appendix E and a short case study in Appendix F.
The activities and time requirements associated with preparation and delivery of the pilot course has
been broken down by task and category of staff involved as follows:
Task
By
Liaison with REHIS re course delivery

and e-assessment
Tutors
Time taken
10
11
Task
By
Time taken
BBB System familiarisation
Tutors
BBB System familiarisation
Learning Technologists
User acceptance test
Learning Technologists; Tutors
Course design and development
Tutors; TV Technician
Remote centre visits; liaison
Tutor; Systems Manager
Develop Student Guide
Learning Technologist
Develop Presenter Guide
Learning Technologist
2.5
Pilot course preparation
Tutors
30
Pilot course delivery
Tutors
53.5
Pilot course delivery support
Learning Technologists;
46.5
28
88.5
25
Head of Centre: Community Engagement

Pilot course preparation support
Learning Technologists
E-assessment development
Tutors; Learning Technologists
Expert technical testing & support

Technologist
11
45.75
4.25
Total
30
388
The staff hours deployed by the Learning Technologists and Virtual Learning Support & Development
Technologist demonstrate the crucial role played by those staff in supporting SCL delivery of the pilot
course relative to tutor input. This level of non-tutor involvement would be unsustainable post-project.
It must be borne in mind, however, that both tutor and non-tutor staff supporting and delivering the
pilot course had no peer group or prior institutional knowledge of SCL delivery on which to draw. Had
they known at the outset what they learned in the course of preparing, supporting and delivering the
pilot, the time required to do this would be substantially reduced as the recurrent issues/problems
associated were known and largely resolved by the end of the project.
The Tutors, Learning Technologists and Virtual Learning Support & Development Technologist all
participated in testing BBB functionality and resolving system performance and connectivity issues in
advance of the first course session. On the day, students experienced significant connectivity
difficulties and the BBB itself did not function as anticipated. Students lost an hour at the start of the
course while technical issues were addressed. As technical problems were resolved, tutors and
Learning Technologists learned to avoid or troubleshoot them in later sessions. By the end of the pilot
course, known performance and connectivity issues had been resolved and no loss of time occurred
during delivery as a result of earlier difficulties.
Direct communication between the online and classroom learners emerged as the single most
significant issue for students. Student feedback clearly shows that, while they found delays due to
technical problems frustrating, they recognised and were prepared to tolerate these when they know
they are participating in an innovative pilot programme. Audio feedback problems encountered when
open speakers are used resulted in online students being muted for long periods and unable to hear
classroom based student discussion/questions directly. Interestingly, the classroom students in
particular felt that this was the most significant drawback for the remote learners (see Appendix E).
Establishing audio communication between the classroom and online cohorts therefore became, and
remains, the major quality development issue for the synchronous collaborative learning methodology
moving forward. It should be noted that the Project has established that it is possible to enable direct
communication between classroom-based and online learners, although this requires the
development of the required experience and competence in non-expert users.
Project management
12
The College appointed an external Project Manager as a dedicated project resource whose time
allocation was ring-fenced to maintaining the projects momentum and ensuring that key activities,
project reports /deliverables, budget management, etc were completed.
At its last meeting the SWEET Project Steering Group expressed a view that, given the unanticipated
internal changes over the year (see below) and other work commitments of project team members,
having an externally appointed Project Manager who could focus on the project alone was crucial to
its successful outcome.
Unanticipated factors impacting on the Project
The SWEET Project ran from January to December 2011. When Jewel & Esk College received
notification from JISC in January 2011 that the SWEET Project submission had been accepted and
the funding awarded the College Principal announced a radical review of the Colleges structure and
staffing. In April 2011 the Boards of Management of Jewel & Esk College and Stevenson College
Edinburgh announced plans to merge the Colleges by August 2012, with Edinburghs Telford College
joining the merger consortium in late November 2011.
The changes arising from College restructuring had a significant impact on the SWEET Project Team
in that:
The Head of Faculty: Service Industries and the Head of Learning Resources, both
representing the College SMT on the SWEET Project Steering Group, left employment in
March;
The job role of the Curriculum Leader: Creative Industries & Arts which included overall
responsibility for the technical development and maintenance of Moodle/BigBlueButton,
disappeared at the end of April;
The SWEET Project Director left employment at the end of June;
The Co-Ordinator (Community) who acted as project administrator left employment in July;
The Head of ICT representing the College SMT on the SWEET Project Steering Group, and
the ICT Systems Manager, Project Team member with responsibility for the technical setup
and installation of BigBlueButton left employment in October.
At the point of submission of the project bid to JISC, none of these changes were known to the Project
Team. It is a measure of the resilience of the Project Team throughout an unprecedented year of
change for the College that we have reported a high level of achievement in meeting the aims,
objectives and outcomes within the original project plan.
3.3 What did you learn?

The appendices to this report provide a detailed description and evaluation of BigBlueButton and the
synchronous collaborative learning methodology in operation. We advise readers interested in drilling
beneath the surface of the key points summarised in this section to take time to review the
appendices in depth.
Optimal performance of a BigBlueButton session requires all computers through which participants
join to conform to a standard minimum setup specification. Performance can be affected by:
Differences in operating system in participants PCs (eg Windows XP, Windows 7, Microsoft
Vista);
Whether or not Flash is installed and, if so, using the required version;
13
Disabling of microphone jackpoints and/or sound levels set for headphones and microphone;
Bad echo feedback to all participants from individuals who use external PC speakers rather
than headphones;
2 microphones picking up an individuals voice it is important to check that only one

microphone is registering participants voice or this produces feedback/echo;
Sharing/unsharing microphones and webcams, which can occasionally result in freezing of

sessions;
The installation of JAVA, required for desktop sharing (to enable participants to see files,
demonstrations, film clips, etc which cannot be shown via the BigBlueButton whiteboard);
Type of headphone used by presenter, where use of double earphone significantly reduces
capacity of presenter to hear classroom cohort while communicating with BBB participants;
Low bandwidth connection, which slows transfer of data and reduces audio/video
performance.
The system performance issues encountered are not unique to BigBlueButton. An online review of
BigBlueButton, Elluminate and Netviewer (facilitated by Scotlands Colleges in December 2011 with
participants joining each session remotely from within Ayr, Forth Valley and Jewel & Esk Colleges and
2 home locations) encountered significant connectivity problems for the participants in all 3 colleges.
The most significant connectivity and performance issues encountered with BBB therefore, appear to
be an inherent difficulty with the technology at its current stage of development.
Knowledge of current technical issues, with documented solutions, will enable the FE community to
develop strategies for managing system performance and connectivity issues pending their
remediation through system upgrades.
The provision of direct on-site support enabled our remote online students to complete the pilot
course, but this level of support is not a sustainable solution as BigBlueButton is mainstreamed.
Confidence in using any IT system develops where users direct experience establishes:
A personal level of trust and expectation that the system will operate consistently to the
operational standard required ;
A personal level of competence which enable them to deploy the system effectively to the
task in hand;
A personal level of knowledge which enables them to work within technical limitations,
anticipate problems presented by known limitations and troubleshoot problems deductively
from previous experience and acquired knowledge;
Access to a responsive source of expert technical support, capable of resolving common

difficulties quickly while at the same time effectively transferring sufficient knowledge for the
user to resolve the difficulties personally as they arise again.
At the level of individual system users, where any one of these factors is under-developed, or at worst
missing entirely, personal confidence and personal performance is compromised.
At institutional level, where any one of these factors is under-developed or missing, the institution is
fatally compromised with respect to deployment, take-up and operational dependence on the system.
14
As a general comment, tutors and Learning Technologists expected more consistency across the
Colleges PC workstations. While non-expert system user expectations were not satisfied in
practice, their views about online delivery in practice changed over time from negative to positive as
their confidence in using the technology grew.
Experience gained from the pilot course shows that problems associated with PC workstation setup
and network connectivity available to students can be anticipated and planned for.
Acknowledging that difficulties were encountered, the tutors and Learning Technologists have learned
from their experience. As one tutor commented: Isnt hindsight great. I think that the level of our
understanding of the system and its potential have taken most of the problems away from future use.
When problems did occur during the pilot course, once these were resolved with explanations of the
cause, tutors were then able to adjust their own actions in order to alleviate or work round the
problems.
New users need to develop a sound practical competence in identifying and troubleshooting known
technical problems which are capable of being readily resolved by non-expert users. The institution
therefore needs to:
Invest in staff training and development required to make web-conferencing work for students,
tutors and the institution;
Ensure sufficient technical support is made available to absorb the support requirements of
the new technology; and
Factor this additional overhead into its system deployment and uptake plans.
New and redeveloped BigBlueButton courses should have a documented course delivery plan which
clearly states the operational use and requirements expected of the system. This plan should be
signed off with the BigBlueButton expert technical support team prior to further course development
and expert technical support deployed as required to ensure system functions meet course
requirements.
Each course team should fully test the delivery of a BigBlueButton course, ensuring that it meets a
standard minimum set of quality and functional criteria.
The pilot course only tested the system with a limited number of online learners. The College should
organise a series of tests involving increasingly large numbers of concurrent users within a single
session and increasing numbers of concurrent sessions until the limits of acceptable performance is
reached. Until the system can be developed to exceed this known operational capacity, all
BigBlueButton delivery should be scheduled within known limits to ensure performance meets
required standards and at worst avoid complete system failure caused by exceeding known limits.
Tutors reported that their experience of the pilot demonstrated that the back up of Learning
Technologists is certainly required at this stage.
Significant time was lost in the first 2 pilot course sessions while difficulties with remote student
connectivity and system performance were resolved. By the end of the 3 rd pilot course these
problems had been largely resolved. One of the tutors commented that the knowledge of connectivity
issues learned from the pilot will enable them to deliver an achievable set of expectations to the future
remote and in house learners.
3.4 Immediate Impact

Rollout development plan
The Jewel & Esk College Principalship has instructed the SWEET Project Director to prepare a
BigBlueButton development plan to go for review and approval at its meeting of 6 December 2012.
The plan will allocate responsibilities for each new area of development with timescales for
implementation from January 2012 onwards. The uses envisaged for BigBlueButton emerging from
the SWEET Project include:
15
College closures. (bad weather etc.) This will enable individual to large group tutorials on
line with tutors/lecturers to enhance and support work sited on discover.
Additional enrolments/participants. Where physical resources are full, additional learners

to be serviced via BigBlueButton with the existing event, increasing opportunities for both
learning and, in commercial courses, revenue.
Remote Learners. Remote learners to access classroom sessions through BBB allowing
higher efficiencies in numbers and revenue without additional use of resources.
Tutorial support. Tutorial support to be delivered to open learners, learners not in

college/class or during period of student absence.
Work Based Learners. Assessors and mentors to deliver and receive communications,
written work, arrange assessment opportunities, speak first hand to expert witnesses
regarding assessments or view actual assessments through BigBlueButton and record it as
evidence. The version of BigBlueButton used in the Project does not support recording of
web-conferencing sessions but this function is currently under development by the Open
Source Project which created the system.
Assessor/verifier Training. Assessors to view and assess feedback to candidates by

trainee assessors/verifiers live using BigBlueButton and give feedback to the trainees without
incurring the time and expense associated with site assessment visits. This would allow the
College to take on remote assessor/verifier candidates, adding value through increased
efficiency and income.
Schools. School pupils to infill into specific parts of existing provision via BigBlueButton from
their school, alleviating the cost pressures of transport and increasing the possibilities of a
mixed provision which could increase the pupils options (more choices, more chances).
Meetings. Allowing remote attendance at internal, cross site or external meetings, saving
time and costs of travel and contributing to green targets. Guest or subject speakers could
giving short presentations to meetings without having to be physically present, but seen by all
(eg, SMT).
Student Access. To extend and enhance the remote learning available to students with the
possibility of pre-recorded sessions via BigBlueButton on Moodle or the possibilities of joining
a similar class via BigBlueButton to catch up on missed learning.
International. International meetings to be conducted via BigBlueButton, sharing uploaded

materials for joint viewing/discussion/negotiation while retaining face to face contact.
Online Assessment. The real possibility of Jewel & Esk College becoming the only centre
able to offer the REHIS Elementary Food Hygiene assessment online is based on the ability
to see and hear the candidate during the assessment as a second level of invigilation. This
would open the possibilities for other awarding bodies who are sceptical of the security and
reliability of online formal assessment.
Inter-campus delivery. Jewel & Esk College is in merger negotiations with Stevenson
College Edinburgh and Edinburghs Telford College with a view to merger being completed by
October 2012. BigBlueButton will support delivery of courses simultaneously to students/staff
in all 3 campuses post-merger, reducing the requirement for locally based staff/students to
travel between campuses
16
Professional engagement. Linking experienced & professional practitioners/managers in

the workplace to students in class for discussion, practical demonstrations, presentations, etc.
Invigilation of remote assessments

A fleshed out methodology for using BigBlueButton to support simultaneous invigilation of student
assessments in distributed physical locations has been produced for the Royal Environmental Health
Institute for Scotland. The Project Team is confident that REHIS will endorse the methodology, which
sets a precedent for other awarding bodies to follow. The methodology has been written in such a
way that it can be readily adapted in College to accommodate invigilation of different forms of
assessment. (see Appendix F).
Staff development and training
The tutor SCL competences checklist developed over the course of the SWEET Project is supported
by:
Published tutor guidelines;
A case study describing the REHIS SCL course delivery methodology;
Central expert technical support; and
Learning Technologists who can use the experience and knowledge of supporting SCL
delivery to extend this support to other tutors and BigBlueButton users.
BigBlueButton itself will be used to deliver simultaneous staff development and training to other
Colleges across Scotland.
National centre of expertise
Access to web-conferencing is available from Scotlands Colleges, the umbrella organisation with a
remit to support the development of the Scottish FE sector. The SWEET Project has now established
a centre of expertise in Jewel & Esk College which, following merger with the other 2 Edinburgh
colleges, will become one of Scotlands largest and leading FE institutions.
3.5 Future Impact

The Project provides the Scottish and wider UK Further and Higher Education sectors with a baseline
evaluation of:
BigBlueButton/Moodle as an affordable integrated web-conferencing/VLE system for any

educational institution considering developing and investing in the technology;
Institutional requirements for successfully establishing and deploying the technology;
The potential for using the technology in support of work-based learning, together with
practical issues concerning learner connectivity and support.
Further Education in Edinburgh

Merger plans between Jewel & Esk College, Edinburghs Telford College and Stevenson College
Edinburgh are already well advanced, with the expectation that the 3 Colleges will merge in October
2012 to become one of Scotlands largest FE institutions. With 2 of the 3 existing Colleges already
Moodle users it is likely that this VLE, together with BigBlueButton will be adopted by the new College.
This would establish BigBlueButton/Moodle within the Further Education service for Scotlands capital
city, establishing a centre of expertise accessible to the national FE/HE system.
Scottish Government and regional coherence
17
Regional coherence between the further and higher education sectors is now emerging as a key
element of the Scottish Governments educational and training strategy. Further and higher education
institutions serving Scotlands regions will be expected to work in closer partnerships, with increased
levels of student articulation and joint delivery of education and training programmes at regional level.
Workbased learning is also emerging as an area for strategic development within this new setup.
The region served by Jewel & Esk College currently sustains a Regional Articulation Hub partnership
comprising all 5 HEIs and all 9 FE Colleges in Edinburgh, Lothians, Borders, Central Scotland and
Fife. The need for higher levels of communication and programme delivery across this partnership
raises the profile of web-enabled programme delivery.
The SWEET Project therefore establishes within the FE regional partnership the web-conferencing
technology, experience and workbased training delivery methodology. As the regional partnership
develops we would expect its more innovative cross-institutional and cross-sectoral programmes to be
supported by the SWEET technology and informed by the practical knowledge and experience
gained from the project.
National networking and support
The Project Team delivered a presentation on its e-assessment methodology and demonstrated the
system to interested delegates at the joint Scottish Moodle User Group/Scottish E-assessment Forum
launch event at the University of Strathclyde in November 2011, attended by representatives from 17
FE and 5 HE institutions in Scotland. Two further UK HEIs, not represented at this event but seeking
to establish an open source web-conferencing/VLE service, have also contacted the project for advice
on BigBlueButton implementation.
The Project has also built a relationship with Scotlands Colleges, through which we have been able to
review alternative web-conferencing systems and establish contacts with Ayr and Forth Valley
Colleges who are also piloting the technology.
It is clear from our own experience (through delivery of the SWEET Project pilot course to remote
learners, reviewing Elluminate and Netviewer as participants and discussion with other colleges using
this technology) that difficulties with establishing remote user connectivity is a current general issue
for managing course delivery and the online learner experience.
The Scottish FE and HE sectors will need to come together in tackling this issue, in particular sharing
effective practice and technical expertise. JISC, in association with Scotlands Colleges and the
Scottish Moodle Users Group, is the natural forum for bringing practitioners together within a national
collaborative network.
The SWEET Project evaluation, practical advice made available through the project documentation
and access to project website/participants should prove a significant early contribution to the Scottish
FE community.
Conclusions
General conclusions
1. While commercially available web-conferencing systems may offer a wider range of user
features (eg emoticons, audio/video streaming from YouTube), BigBlueButton offers
educational institutions a roughly equivalent range of essential functions without licence fee
costs.
2. Deployment of BigBlueButton does, nevertheless, incur initial hardware costs (server,
headsets, webcams) and the investment of significant levels of staff time associated with
expert technical support, staff training, course re-development and learner support for BBBenabled course delivery.
18
3. The most significant connectivity and performance issues encountered with BBB therefore,
appear to be an inherent difficulty with the technology at its current stage of development.
4. Establishing audio communication between the classroom and online cohorts therefore
became, and remains, the major quality development issue for the synchronous collaborative
learning methodology moving forward.
Conclusions relevant to the wider community
5. Simultaneous delivery of training courses to workbased and on-campus student cohorts using
the synchronous collaborative learning methodology can be made to work pedagogically, but
requires employers to ensure that the locally available computer workstation setup meets a
specified minimum software, connectivity and audiovisual hardware requirement.
6. The provision of direct on-site support enabled our remote students to complete the pilot
course, but this level of support is not a sustainable solution as BigBlueButton is
mainstreamed.
7. User confidence in using IT generally and a web-conferencing system specifically is crucial to
the success of the SCL model. Where any of the 4 key criteria for establishing the required
levels of user confidence from personal to institutional level is under-developed or missing
(see p 14), the institution is compromised with respect to deployment, take-up and
operational dependence on web-conferencing.
8. In the context of the SWEET Project, our experience shows that technical problems
encountered in College with BigBlueButton can and should be resolved through timely and
effective solutions provided by ICT and BigBlueButton expert in-house technical support.
9. Of the 25 students who completed the final examination, 24 (96%) passed. In terms of
student achievement this is commensurate with the traditional face to face delivery model.
There was evidence of lower pass marks which tutors ascribe to time lost due to early and,
post-project, known, manageable and largely non-recurrent technical issues. The SCL
delivery model therefore resulted in equivalent pass levels in the course assessment, with
lower pass grades which tutors expect to raise to previous levels as SCL delivery beds in.
Conclusions relevant to JISC
10. The Scottish Funding Council is responsible for developing regional coherence within the
Scottish tertiary education system, bringing together colleges and HEIs within a collaborative
and rationalised framework of post-school training and education within regional hubs. This is
resulting in a range of approaches, from formal and informal inter-institutional agreements
and arrangements to full institutional mergers. Where courses require to be delivered from
multiple institutions, potentially at some distance from one another (and the learners),
adoption and use of synchronous web-conferencing learning systems offer a workable
logistical solution to course delivery issues arising from geographical location and timetabling.
11. BigBlueButton works in an FE context and, in the context of the SWEET Project was made to
work within a challenging course delivery methodology.
12. There is significant interest in open source web conferencing applications within the Scottish
FE sector, with colleges actively seeking advice and support. JISC is the national agency for
coordinating and delivering this support.
19
Recommendations
General recommendations
1. Jewel & Esk College should continue to deploy BigBlueButton, using the evidence of the
SWEET Project as presented in the final report and other project documentation to inform a
full deployment and development project. Sections 3.1 (Project outcomes, outputs); 3.4; 3.5
(Further Education in Edinburgh)
2. Jewel & Esk College, should conduct a BigBlueButton stress test which establishes the
maximum capacity of the current system and ensure that its system development plan
operates within operational parameters. The results of this test should be submitted to JISC
and disseminated via the SWEET Project website. Section 3.3 p14
3. Colleges and HEIs planning to develop a web-conferencing system which will support course
delivery, learner support and other forms of work-related communication should adopt a
managed project methodology with clearly defined aims, objectives, responsibilities and
resources underpinned by a realistic development plan. The project manager should report to
a project director at senior manager who commands the authority to effect institutional
change. Section 3.2, Project methodology p7
4. A key procurement consideration in the choice of web-conferencing system should be its
operational dependence on third party software (eg Flash, Java, operating system) being
installed on user workstations and whether the system itself requires to install any software on
user PCs. This could have major implications for the current institutional workstation software
specification, the operational costs of updating software across networked PCs and network
security policy and present connectivity issues for external users from workstations beyond
institutional control. Section 3.3, para2, p13-14
5. Prior to procurement and deployment of the web-conferencing system, colleges should
conduct a full audit of networked workstations and ensure that these support full connectivity
and use of the system. Section 3.3, para3-4, p13
6. Colleges should require staff directly involved with web-conferencing course delivery develop
a minimum level of competence in both technical and pedagogical aspects of system use and
the design, management and delivery of synchronous web-enabled training. Section 3.2
(Phase 5, p11 -12; Appendix H
Recommendations for the wider community
7. The Scottish Funding Council should consider the allocation of funds to support the adoption
and development of web-conferencing at regional level in support of the regional coherence
agenda. Section 3.5 p17-18
8. In their promotion of work-based learning programmes delivered via web-conferencing,
colleges and HEIs should ensure that customers are made fully and clearly aware of the local
technical setup and support required before committing to enrol their staff on training
programmes. Section 3.2, p12; Section 3.3, para3-4, p14; Appendix C
9. Scotlands Colleges, the Scottish Moodle Users Group and JISC should jointly develop a staff
training and development progamme in web-conference supported course delivery. The
programme should be accessible online and use the technology itself to develop practioners
within the FE and HE communities. Section 3.4 (Staff development and training), p17
Recommendations for JISC
10. JISC should continue to promote use of BigBlueButton as a workable web-conferencing
system within the FE and HEI community. Section 4 (Conclusions relevant to JISC) p19
20
11. JISC should review recent projects which have piloted web-conferencing course delivery
methodologies and use the project reports and deliverables to develop a practical set of
management, support staff and practitioner guidelines for colleges and HEIs, supported by
specifications, evaluations, guides, processes, checklists etc which lend themselves to
general adoption by other institutions. Section 3.5 (National networking and support) p18
Implications for the future
Building on the SWEET foundation

JISCs support of the Sweet Project has enabled the development of practical resources (SCL delivery
model, checklists, user guides, evaluation tools, detailed pilot project evaluation, student evaluation
tools, e-assessment methodology and associated materials, employer checklist) and an initial base of
technical and course delivery expertise upon which other educational institutions can draw in planning
and implementing their own BigBlueButton or other web-enabled conferencing systems.
The Synchronous Collaborative Learning methodology applied to the project pilot course has been
developed into a generic model and documented within the Project outputs (see Appendix G). The
model has considerable development potential, particularly in relation to its underlying pedagogy.
Other course tutors who adopt, adapt and develop the SCL methodology should develop the initial
SWEET model, document this and disseminate it to the wider user community via the channels
identified below and to the long-term Project contact for adding to the Project website.
Sustainability
Jewel & Esk College is committed to developing BigBlueButton as its institutional web-conferencing
system. As its user base develops, so too will the College as a centre of expertise. The SWEET
Project outputs represent the initial pool of SCL and BBB resources from which new users can draw
and to which they can contribute and extend.
In the spirit of open source, all resources developed in the course of this project (with the exception of
REHIS-specific course materials) are being made openly accessible to the wider educational
community. Other institutions can adopt (and adapt as required for their own purposes) freely.
Where existing Moodle/BBB users have the system developer capacity to create new BigBlueButton
modules, functionality or generic BBB features, these could be shared with the BigBlueButton. Open
source project (see below) for incorporation into the main development system.
User community
The JISC Regional Support Centre Scotland already hosts the Scottish Moodle User Group (SMUG) a
forum for exchanging information and expertise. SMUG could be extended to embrace BigBlueButton
until a critical mass of user institutions has developed, at which point a separate BBB Forum could be
established to take forward dissemination and development work.
The BigBlueButton Open Source Project (see reference below) provides an existing forum for expert
technical developers. UK educational institutions which deploy ICT developer expertise to the
development of the underpinning BigBlueButton system modules/code should participate in Project
groups and actively contribute to the development of system functions which better meet UK
educational requirements.
Long term Project contact
For information and assistance with queries the longer term SWEET Project contact is:
21
Wendy MacAdie
Head of Centre: Community Engagement
Jewel & Esk College
Edinburgh Campus
24 Milton Road East
Edinburgh
EH15 2PP
Tel: 0131-344-7050
Email: WMacAdie@jec.ac.uk
Note: As at January 2012 Jewel & Esk College is in merger consultations with Stevenson College
Edinburgh and Edinburghs Telford College. Merger is planned for October 2012, at which point the
College name and contact information will change.
References
BigBlueButton website. Tutorial videos. http://bigbluebutton.org/content/videos

Short orientation videos for tutors and system developers.
BigBlueButton website. Support. http://bigbluebutton.org/support
Access to instructions and support for installing BigBlueButton; FAQs; system user groups (Setup,
Users and Developers); Developer support.
BigBlueButton. Open source project. http://code.google.com/p/bigbluebutton/
Website for the BigBlueButton open source system development project. Provides access to a range
of resources, links, groups for open source developers who add or develop modules, functions code.
Includes detailed system installation resources
.
Appendices
APPENDIX A: GLOSSARY OF ACRONYMS USED IN REPORT

APPENDIX B: COMPLETED BBB USER ACCEPTANCE TEST REPORT
APPENDIX C: BIGBLUEBUTTON: RECOMMENDED COMPUTER SETUP FOR OFF-CAMPUS
USERS
APPENDIX D: REHIS PILOT COURSE EVALUATION REPORT
APPENDIX E: REHIS STUDENT COURSE EVALUATION SUMMARY REPORT
APPENDIX F: SYNCHRONOUS COLLABORATIVE LEARNING CASE STUDY
APPENDIX G: SYNCHRONOUS COLLABORATIVE LEARNING: COURSE PLANNING, SETUP
AND DELIVERY MODEL
APPENDIX H: SYNCHRONOUS COLLABORATIVE LEARNING / BIG BLUE BUTTON TRAINING
SPECIFICATION: COURSE TUTOR COMPETENCES
22
APPENDIX A: GLOSSARY OF ACRONYMS USED IN REPORT

BBB BigBlueButton open source web-conferencing application.
FE Further Education
HE Higher Education
HEI Higher Education Institution
JISC Joint Information Systems Committee
REHIS Royal Environmental Health Institute for Scotland
SCL Synchronous collaborative learning
SWEET Synchronous Web-enabled Employee Training
SMUG Scottish Moodle Users Group
VLE Virtual learning environment
23
APPENDIX B: COMPLETED BBB USER ACCEPTANCE TEST REPORT
User Test of Big Blue Button Features
Test
Teacher log on
Student Log on
Yes
Yes all students names start the same so

impossible to tell who is who in the chat
Presenter
Check audio works
See each other - Check webcams
Upload Presentation
Presentation features Zoom, White board
Share your Desktop
Mute or Eject Participants
Delay is annoying. If you have your headphones

on and you are a listener and you share your
microphone you hear what you have just said
after a short delay which makes concentrating on
speaking difficult.
Yes
Yes
Yes
Poorer quality but OK. Can be minimised but then
difficult to see or remember where it is.
Worked after what seemed like a long delay.

Need to test again.
Viewers
See each other - Check webcams
Public and Private Chat
Yes
Can only see first few letters of username

would be better if we could see whole name.
24
See all Participants
Raise Your Hand
See the Presenters cursor
Watch presentation and zoom
Yes webcams work on Windows 7 without

needing to install software. Sometimes view
hidden under presentation.
OK but not obvious when it happens
Yes
Presentation is difficult to see as it cuts out a lot
of formatting. Changed background to white
which meant you couldnt read the white text!
Presentation was converted to a PDF which
worked better than a .ppt.
A student cannot zoom just the presenter.
Audio / Mute facility
Delay annoying but may be better when not in

same room. Annoying when you hear your own
voice delayed when presenting need to turn off
being a listener when presentation and rely on
students raising a hand.
Notes
Set up time 25 minutes.
Managing the screen with multiple boxes difficult. Maybe just use webcam to see presenter and then
just maximise the others if you want to speak to them.
Need to go over ground rules at the beginning. E.g. raise hand to ask a question.
How do you know who is having a private chat and not listening?!
Can see it working well with a presenter here and various individuals watching and listening from
different places outside the college but we are struggling to see how it would work if the presenter was
teaching a class here whilst others were tuning in both practically (sitting in front of camera) and
pedagogically.
25
APPENDIX C: BIGBLUEBUTTON: RECOMMENDED COMPUTER SETUP FOR OFF-CAMPUS

USERS
PC hardware
Computer requirements
Logging on to and using BigBlueButton requires users to have access to a PC with a web browser as
a minimum.
The more internal memory the PC has, the better its web browser will perform.
If using the Windows 7 operating system, Microsoft recommends at least a 1Ghz processor and a
minimum of 1GB of memory.
Please note that those are minimum requirements. The faster the processor and larger the internal
memory available, the smoother the computer workstation will run BigBlueButton.
Monitor
The bigger the monitor the more easily the user can move and reorganise the BigBlueButton desktop,
but as long as users can view web pages comfortably they will be able to use BigBlueButton.
Web Cam
A web camera is desirable to allow online users to see each other. If used, a low resolution webcam
will be sufficient as users will only been seen on-screen at a 320x480 resolution.
Headset
BigBlueButton users require a comfortable audio headset with:
Earphones which output a clear audio signal; and
Microphone which transmits clear voice communication.
Use of external speakers during an online BigBlueButton session should not be permitted as this can
cause significant loss of audio signal quality.
Internet connection
A 2 Mbps broadband connection is recommended as a minimum for internet use.
Bandwidth available to BigBlueButton users is the most important factor for ensuring a fast and
reliable network connection to the system. The faster the local network connection available to remote
users the better their interaction with other BigBlueButton users will be.
If only a slower broadband connection is available to users, they should ensure that they do not:
Download any files or software; or
Stream any other videos while using BigBlueButton to ensure that the remote connection
performs as smoothly as possible.
26
PC software
Web Browser
We recommend Mozilla Firefox for the best user experience with BigBlueButton. If not already
installed, this application can be downloaded and installed from the internet free of charge.
Other popular web browsers such as Internet Explorer and Google Chrome are also compatible with
BigBlueButton.
Adobe Flash
The latestst Adobe Flash plugin should be installed on the user PC web browser. Without this
software, users will not be able to participate in BigBlueButton web-conferencing sessions.
We recommend that users install the latest version of Adobe Flash which can be download free of
charge from www.get.adobe.com/flashplayer
Important:
Users who need to install software on their computers in order to meet our
hardware and software recommendations will need administrator rights on their PC to do this
personally. If in doubt, or if using a workplace computer users should seek advice and support local
ICT technical support staff or call the Jewel & Esk College Helpline
27
APPENDIX D: REHIS PILOT COURSE EVALUATION REPORT

SWEET PROJECT
BigBlueButton REHIS pilot course evaluation

The evaluation should aim to establish what worked and what didnt work well for the students,
lecturers, Learning Technologists and ICT technical support staff.
The project team will have a range of evidence to consider:
Direct experience of the tutors and Learning Technologists involved;
Student questionnaire and focus group feedback;
Recording of the classroom session;
Recording of the whiteboard during the session (Jane to confirm).
Input received from:
Ron McGilp
Jane Robertson
Annette Allison
Jarod Carruthers
Wendy MacAdie
Donald Steele
Jan Crawford
REHIS pilot course students
28
A. BIGBLUEBUTTON FUNCTIONALITY
A1. What functional issues emerged during BBB course setup and testing and how were these
resolved?
PC setup and connectivity
Optimal performance of a BBB session requires all computers through which participants to conform
to a standard minimum setup specification. Performance can be affected by:
Differences in operating system in participants PCs (eg Windows XP, Windows 7, Microsoft
Vista);
Whether or not Flash is installed and, if so, using the required version;
Disabling of microphone jackpoints and/or sound levels set for headphones and microphone;
Bad echo feedback to all participants from individuals who use external PC speakers rather
than headphones;
Sharing/unsharing of microphones and webcams which can occasionally result in freezing of

sessions.
Desktop sharing permissions, required to enable participants to see files, demonstrations, film
clips, etc which cannot be shown via the BBB whiteboard;
Type of headphone used by presenter, where use of double earphone significantly reduces
capacity of presenter to hear classroom cohort while communicating with BBB participants.
Low bandwidth connection, which slows transfer of data and reduces audio/video
performance.
Solutions
1) Establish minimum setup and software specification for all college computers, including
desktop sharing, and ensure that the specification is maintained.
2) Provide the minimum technical specification to all external participants as far in advance of
the session as possible and advise participants to either check and confirm their PC meets
the specification, using local technical assistance where required.
3) A Troubleshooting guide was produced by the Project Team. Make this available to all BBB
users and update the guide as new issues are identified and resolved.
4) Arrange a pre-session induction and setup check for participants.
5) Consider establishing a technical helpline providing expert technical support with capacity to
take over local PC to bring setup to minimum specification.
6) Check for usage of desktop speakers prior to session and ensure that either the participant
switches to headphones or remains muted until s/he indicates a request to speak.
7) Provide presenters who use BBB to join external participants to a face-to-face classroom
session with a single earpiece headset.
8) Mute participants microphones during sessions and unmute/mute individuals as required
during discussion.
29
Provision of outreach technical support

In order to ensure that the local PC configuration and connectivity met the minimum technical
specification required for remote access by learners provided from Project partner employers, the
Project deployed expert technical support to check the local setup prior to students participating in the
pilot course. Where problems emerged during the course in one centre a Learning Technologist was
deployed to provide on-site access to 3 College laptops specially configured for BBB together with
personal local troubleshooting support.
The provision of this level of direct on-site support enabled the remote students to complete the pilot
course, but is not a sustainable solution as BBB is mainstreamed. Where a remote centre enters into
an agreement/partnership with the College to guarantee a minimum number of student enrolments on
BBB-delivered courses, the College may consider sealing the agreement with an initial local computer
setup check/upgrade and a telephone/BBB helpline service.
A2. How confident were the tutors & Learning Technologists of BBB functionality prior to the pilot
session?
BBB sells itself to users as being an operationally simple system to use. The short user videos
accessible on the BBB website show presenters and students how to use the functions available to
them during web-conferencing sessions. Tutors and Learning Technologists opted not to receive
formal training in using the system as the functions appear straightforward to learn and understand
as indeed they are in a robust and reliable technical operating environment.
The Project agreed to adopt an approach to BBB user induction which required the Learning
Technologists to prepare non-expert user acceptance test criteria and apply the test which had 3
principal objectives:
Establish how quickly non-expert users could learn how to use the system without immediate
expert support;
Put BBB, which had already passed an expert technical acceptance test during system setup,
through a typical user acceptance test.
Enable the Tutors and Learning Technologists to acquire the level of functional competence
required as presenters to deliver and support BBB courses.
The Tutors and Learning Technologists tested BBB on several occasions prior to the first pilot course
session and encountered intermittent/recurrent system performance problems as outlined in A1
above. Typically, non-expert acceptance test sessions would proceed to the point where a significant
problem required resolution by experts before further testing could continue. As they became more
familiar with the system, seeking expert technical support to resolve system performance issues, their
confidence as users grew.
Confidence in using any IT system develops where users direct experience establishes:
a personal level of trust and expectation that the system will operate consistently to the
operational standard required ;
a personal level of competence which enable them to deploy the system effectively to the task
in hand;
a personal level of knowledge which enables them to work within technical limitations,
anticipate problems presented by known limitations and troubleshoot problems deductively
from previous experience and acquired knowledge;
30
access to a responsive source of expert technical support, capable of resolving common

difficulties quickly while at the same time effectively transferring sufficient knowledge for the
user to resolve the difficulties personally as they arise again.
At the level of individual system users, where any one of these factors is under-developed, or at worst
missing entirely, personal confidence and personal performance is compromised.
At operational team level, access to expert support from one or more peers is an effective method of
developing individual competence within the peer group and resolving problems as they occur.
At institutional level, where any one of these factors is under-developed or missing, the institution is
fatally compromised with respect to deployment, take-up and operational dependence on the system.
Within the SWEET Project, system setup and expert testing took 10 days over a period of 2-3 months.
At this stage the system was handed over to the non-expert users as having passed the expert
technical tests and in full working order. Non-expert user testing, take-up and deployment of BBB to
deliver the pilot course took approximately 400 hours over 6 months from the point of system
handover to delivery of the first live session. At this stage, the prior non-expert user testing had
established that, under test conditions, the system operated to the functional standard required to
deliver the pilot course.
Given the difficulties experienced during the non-expert system testing phase while Tutors and
Learning Technologists saw for themselves that BBB functioned to requirements, there was sufficient
evidence of technical performance problems which reduced their level of confidence that the system
would be guaranteed to work as required within the pilot course.
Where the non-expert users encountered recurring functional problems they required:
a)expert and timely technical support required to identify and resolve the problems;
b) The transfer of sufficient operational competence and knowledge required to troubleshoot

the problems as they might occur subsequently.
While expert technical support was available and provided the non-expert users developed the
perception that the problems encountered were recurrent and that they lacked adequate expert
support pending development of the personal knowledge and competence required to troubleshoot
them should they recur. As it was difficult to anticipate when and where the problems might recur, the
default response adopted was not to use particular functions where these had been followed by
functional failure as the one became associated with the other. This impacted on the quality of
course delivery (eg where a video specially recorded for the course was not played on the shared
desktop for fear that this would freeze the system as had happened on a previous occasion).
Having encountered functional problems and entirely lacking the peer support which naturally
becomes available for well- established systems, the non-expert users struggled to overcome these
but, through trial and error and provision of expert support developed a level of functional competence
which led to marked improvements in course delivery and student feedback. In the words of one
tutor, by the end of the pilot course the non-expert users had become relatively confident as we have
come to understand the systems idiosyncrasies and work round them.
An alternative view expressed from within the Project Team was that the user testing was not robust
enough and . . . should have been more extensive while a third non-expert user view at the end of the
pilot was that the tutors were confident using the system as were the Learning Technologists
Lack of personal confidence in technology will carry over to behaviour, body language, course delivery
during the course in general. Additional non-expert user support is required.
Solutions
31
1) Prior to initial course delivery new tutors need to practice with system until they are perfectly
confident with all functions.
2) Introduce an online practical tutor test where peers adopt the role of students, make all the
common mistakes for logging on and setting up and require the non-expert. Tutors need to
be able to solve known problems/pitfalls in situ and be able to diagnose common problems
quickly and with confidence.
3) Continue to log all known functional problems, their causes and non-expert solutions and
make this accessible via Discover.
4) Provide a non-expert user training course covering system functions, trouble-shooting and
course planning/development delivered via BBB.
A3. How did tutors and Learning Technologists expect online delivery to function?
As a general comment, Tutors and Learning Technologists expected more consistency across the
Colleges PC workstations. In practice a known minimum standard specification across all networked
PC and laptop computers enables users to switch between workstations when required with
confidence that they will all conform to minimum technical specification requirements.
Individual perspectives voiced ranged from It should work - it didnt following the first online course
session to Difficult to know what to expect until we did it for the first time. I did expect a few glitches
but once all learners could hear and take part it functioned as we had hoped at the end of the trial.
A4. How did online delivery function in practice?
While non-expert user expectations were not satisfied in practice, their views about online delivery in
practice changed over time from negative to positive. A tutor view expressed following the second
online session reflects the frustration felt at loss of time due to technical issues and the impact of this
on the class:
Over the two sessions we lost approximately 1.5 hours of input due to technical failures.
While we managed to catch up and cover all the areas of the course in the remaining time,
it was rushed and the full value and extent of the learning was diminished. This was
evidenced by the level of exam results of the traditional learners. While all three passed the
examination the level of the passes, based on my experience and my feel of the learners
ability, was significantly lower than I would have expected. My own view is that we
sacrificed the stories and anecdotes that assist learners to understand and retain the
knowledge in order to cover the required content.
In contrast, tutors views had become significantly more positive by the end of the 3 rd pilot course, as
illustrated by the following comments:
Better and better as we came to terms with its limitations and understood the issues better.
Once the problems were resolved the sessions generally went fine except for minor issues. We were
frightened to click on anything after it started in case we ended the session prematurely.
It is almost axiomatic that new ICT systems fail to meet functional expectations in the initial stages of
deployment, despite pre-deployment acceptance test results which indicate otherwise. From a users
perspective, initial disappointment and frustration can be relatively quickly turned around through
system familiarisation and remediation of technical issues.
32
B. COURSE REDESIGN
B1. How was the traditional REHIS course design altered in order to adopt synchronous collaborative
learning? How well did this work in practice?
The traditional REHIS course was delivered in two 3 hour sessions over one day, with a formal
multiple choice examination at the end of the course on the same day. Tutors reinforce learning using
relevant but surprising stories and anecdotes which also serve to lighten the experience of an
otherwise intensive training day. The syllabus is fixed by REHIS who also provide a course workbook,
Powerpoint presentation and formal end of course test requiring formal invigilation. Given the need to
conform to REHIS requirements, there was little scope for changing the course syllabus.
The tutors broke the delivery of the 2 course sessions over two consecutive Mondays, allowing
students to reinforce the learning from session 1 and prepare for session 2 over the course of the
intervening week. They also reviewed the Powerpoint presentation and substantively changed this in
order to better meet the needs of both the classroom and remote students.
Solutions
1) The course structure was redesigned for delivery over 3 X 2 hour sessions, recognising that
while this should improve the experience for the remote learners it did mean that the in house
learners were required to attend on three occasions.
2) During the course of delivering the 2 sessions it became evident from the online students
demeanour and body language that 3 hours was too long for remote students to retain
interest and concentration. Tutor presentation methods were modified to create a more
interactive environment with both types of learner. Feedback from one of the tutors one of the
tutors following the first pilot course is revealing:
This improved visibly in the second session as we, as presenters started to naturally adapt to
the new learning environment. We need to look at the course again and modify the resources
and plans to accommodate these changes more effectively and improve the learning
experience
The use of more directed questions and asking specific learners for their current experiences
of certain elements rather than the more general approach to the learners will help us to
create a more equal level of involvement and ownership of the course by both types of
learners.
We need also to look at the potential interaction between the remote learners on chat to
carry out short collaborative exercises at the same time as in house learners and the
possibility of handing over the headset to enable an in house learner and remote learner to
discuss and report back on exercises.
While these seem simple changes to introduce they will require a review of the course
delivery as any of these suggestions will require very specific exercises and classroom
management to work properly within the available time if they are to enhance the learner
experience rather than create another time issue. None of these adaptions are possible until
the technical side of blue button is totally
reliable or we would create more issues than
improvements.
3) The tutors decided to limited use of group exercises and group work which were more
appropriate to classroom cohorts in favour of much more open forum activities and directed
questions. This enabled the remote learners to participate more directly within the class but
required the tutor to act as a voice conduit between attending and remote learners in the
absence of an open microphone in the classroom.
33
B2. What practical and pedagogical opportunities/problems were anticipated and how were these
planned for and accommodated within the overall course redesign?
The approaches to pilot course redesign reviewed in B1 above are also relevant here. The Tutors
allowed for a more limited level of learner input and removed of group exercises and activities to allow
for open forum activities and directed questions.
Given the additional time required to manage delivery to classroom and online learners
simultaneously the Tutors placed additional learning resources on the Colleges Discover VLE for
students to access and use between sessions as preparation and revision to underpin the learning.
B3. How far did the anticipated course redesign outcomes match what actually happened during the
course sessions?
By the end of the pilots most of the expectations were met apart from the audio issues which were
easily dealt with and the desk top sharing issue.
B4. How could any unanticipated problems be better identified and planned for in future?
Unanticipated problems were technical rather than pedagogical, eg BBB user screens froze and the
BBB server crashed unexpectedly in earlier sessions but once addressed did not happen thereafter.
Sound and echo problems anticipated but proved not so simple to resolve.
Experience from the pilot course shows that problems associated with PC workstation setup and
network connectivity available to students should be anticipated and planned for. Broadband
available to remote students also needs considered (e.g. online students connecting from a centre in
Tranent found that their Library took priority connection and threw them off session; the students
therefore had to use a dongle in order to connect to the BBB, incurring additional expenditure which
the Project covered for the pilot course).
Until resolved internally, problems associated with the Colleges own non-standard PC workstations
will require BBB users to be scheduled into PCs known to meet the minimum software specification.
Tutors should check BBB functionality on scheduled workstations prior to online course delivery.
Acknowledging that difficulties were encountered, the tutors and Learning Technologists have learned
from their experience. As one tutor commented:
Isnt hindsight great. I think that the level of our understanding of the system and its potential have
taken most of the problems away from future use.
Solutions
1) Carry out more extensive testing of the system in advance of early user course delivery,
document issues/problems encountered and solutions to these.
2) Use student online student induction sessions to clearly communicate and minimise, if not
eliminate, common problems associated with local IT facilities.
3) With respect to course delivery, tutors should agree at the outset with online students how
they will participate during the session and engage with learning activities using BBB
communication channels (eg audio, video, chat).
4) While delivering/facilitating online sessions tutors should learn to monitor attempts by learners
to communicate and engage via the communication channels open to them in order to
minimise delays in responding. This will develop with experience but is a key skill to actively
34
develop in tutors from the outset.
B5. What unanticipated benefits arose from the new course design and how can these be built upon
for future courses?
Tutors were surprised at how positive the student feedback was in spite of early technical problems
which interrupted course delivery. This appears to show that when students are made aware that they
are participating in something new in the course of induction, they are more likely to tolerate technical
or pedagogical glitches in the early period of course delivery.
The potential for delivering the on line assessment became clear to REHIS tutors from the outset as
this addressed a course development issue already highlighted within the awarding body. The
methodology developed by the project will be formally submitted to and reviewed by REHIS with a
high expectation that it will be accepted. The methodology is transferrable to other assessment
contexts and provides an opportunity for approaching other awarding bodies, reaching similar
agreements and opening up formal online assessment using web-enabled systems.
The requirement to redesign part of the traditional course in which classroom students went on a tour
of kitchen facilities led to the production of a video tour with commentary which could be shown over
BBB. This proved very successful for both classroom and online students and the video can now be
used in both face to face delivery and made available on the Discover VLE for individual student use.
The additional course resources made available on Discover for use by the learners between course
sessions were very well taken up by the students. The Tutors will now consider adopting this
approach in the delivery of other courses.
C. SYSTEM TESTING / TECHNICAL ISSUES

C1. What specific issues arose during user testing prior to the course delivery days? How effectively
and quickly were these resolved once identified? Could lines of communication and responsiveness
have been improved and, if so, how?
Specific technical issues arising prior to and during the pilot course delivery days included:
Unexpected freezing of BBB session, requiring a reboot of the server;
Poor audio quality from user microphones, arising from previously unknown additional
microphone in the webcam used by Tutors. When discovered, the webcam microphone was
muted and this resolved the problem, highlighting the need for system users to develop a
sound troubleshooting knowledge of such key issues prior to course delivery.
Unexpected failure of presenter to share the desktop.
Unavailability of latest version of Java required to run the video produced for the course.
Requirement to clear the cache when using Java through Internet Explorer; Firefox
subsequently used for course delivery.
Lack of consistent networked PC workstation setup across the College network; Network
Services upgraded individual workstations when required, but the College should agree and
implement a common minimum software specification for all College workstations.
One of the Tutors commented that when problems occurred during the pilot course, once these were
resolved with explanations of the cause, they were then able to adjust their own actions in order to
alleviate or work round the problems.
35
Solutions
1) In the context of the SWEET Project, experience and hindsight - shows that technical
problems encountered in College with BBB can and should be resolved through timely and
effective solutions provided by ICT and BBB expert in-house technical support.
2) In rolling out BBB post-project, the College should develop and implement a system
deployment and support plan which broadens not only the user base but also the expert user
support base.
3) Early users of BBB post-project will face the personal confidence issues identified under A2
above. The College should ensure that its system deployment and support plan is supported,
driven and monitored by the Senior Management Team until a critical mass of users working
within an effective non-expert user and expert technical user support infrastructure exists.
C2. What system testing lessons have been learned for future course planning and delivery?
New users need to develop a sound practical competence in identifying and troubleshooting known
technical problems which can be readily resolved by non-expert users.
New and redeveloped BBB courses should have a documented course delivery plan which clearly
states the operational use and requirements expected of the system. This plan should be signed off
with the BBB expert technical support team prior to further course development and expert technical
support deployed as required to ensure system functions meet course requirements.
Each course team should fully test the delivery of a BBB course, ensuring that it meets a standard
minimum set of quality and functional criteria.
The pilot course only tested the system with a limited number of online learners. The College should
organise a series of tests involving increasingly large numbers of concurrent users within a single
session and increasing numbers of concurrent sessions until the limits of acceptable performance is
reached. Until the system can be developed to exceed this known operational capacity, all BBB
delivery should be scheduled within known limits to ensure performance meets required standards
and at worst avoid complete system failure caused by exceeding known limits.
D. TRADITIONAL V NEW PEDAGOGICAL MODEL
D1. How well did the classroom cohort engage with the new course compared with previous course
delivery methods?
Tutors were very conscious of the disruption to the course delivery caused in the early stages of the
pilot by technical and online communication problems described above and therefore, following
student evaluation stated that they were: were surprised at the positive view they all took.
From the Tutors perspective, the most negative classroom cohort related to lack of direct interaction
between the remote learners and themselves, which was not possible due to the inability to link the
classroom up with an open table-top microphone.
A Learning Technologist who also attended the early sessions commented: I was most impressed
with how quickly Nan and Ron adapted to teaching local and remote user and making sure to engage
both groups by repeating any questions that were being asked in the class to the remote students.
36
D2. What issues for a) tutors; b) Learning Technologists; c) students; d) employers emerge from the
video and whiteboard recordings of the pilot course sessions. How should these be addressed?
Tutors felt that their experience of the pilot demonstrated that the back up of Learning Technologists
is certainly required at this stage.
It was felt that it would improve the remote user experience to mount a webcam to a laptop that could
sit on the table facing the lecturer in order to reduce amount of refocusing the webcam has to do
when a lecturer is very close to it.
D3. What was the anticipated impact of the new methodology on course delivery?
There was broad agreement that the new methodology would address issues of inclusivity for remote
learners by enabling them to take part in live classes.
More broadly, access to BBB impacts course delivery in a number of different ways, including:
In extreme weather conditions students and Lecturers alike can log in and conduct course
sessions where travel to College is disrupted .
Appropriateness to particular course subjects course subjects (eg IT, Core Skills);
Delivery of one to one or group tutorial sessions off-campus.
D4. What was the actual impact of the new methodology on course delivery?
It has allowed remote access to learners although the levels of participation between remote and
learners in the room has been variable. Where poor connectivity is due to the band width available to
the remote centre then, in the interests of the complete group of online learners, those whose local
connectivity impairs the audio quality enjoyed by others will have to accept that their audio will be
muted until requested to speak.
E. STUDENT PREPARATION AND INDUCTION
E1. How were both the classroom cohort and online students prepared for and inducted to the pilot
course?
Online learners received BBB student guidelines in advance of the course. One student ignored
instructions to mute speakers, highlighting the need to have an immediate pre-course login session
where tutor/Learning Technologist checks that participants are all technically set up and
communication works correctly. The webcam link from the students can be used to check if any are
participating without wearing headphones, in which case they should be asked to mute external
speakers if being used.
A checklist of technical and course delivery issues was prepared used to ensure that the induction
received is complete.
The recruitment of students to the first pilot course was organised at very short notice and the College
did not receive contact details until very close to the first session delivery day. Consequently,
induction was patchy depending on how near to start time contact with remote students was made.
This highlights the need for students to be recruited and their contact details supplied to the staff who
will take them through a remote connectivity check and induct them to the course.
Where connectivity problems occurred members of the Project Team went to the places of
work/outreach centres and sat with the learner before and during the course. This worked much
37
better, allowing the visiting College staff member to iron out local connectivity issues. In practice and
under normal conditions, however, would not be a viable option in which case the inducting support
staff would need some time with the learner the previous week to ensure their set up was correct and
they knew how to log in.
E2. How well did the preparation and induction process enable the students to adapt to the different
course delivery?
Tutors believe that the initial technical problems encountered and late notification of student details
meant that the induction provided did not address the issues encountered very well. Following the
pilot, however, they feel that their experience in using the system has enhanced their understanding
and knowledge of what is required in using the system, placing them in a position to improve this
considerably.
From a Learning Technologists perspective induction went well. Students were sent the guide to read
prior to the class. An online overview of the screen was given and explained opening new
windows/tab emphasised as it was to logout of the session. We built on the information as we went
through the trial. Students were comfortable with the online experience and said they enjoyed this
form of learning they did not like the sound issues though at the beginning.
From feedback received by a Project Team member who was present in a local centre during one of
the sessions, one student felt disengaged and said Ill just observe rather than participate as
attempts to chat on a social, and then practical way (in an attempt to help with the issues with another
students machine at Tranent), went completely unnoticed. As the class started he tried to use chat
again to engage and it was not picked up.
E3. What worked well and what improvements could be made to the induction methodology, the
enrolment process and the level and scope of information provided?
The Learning Technologist involved with most remote student induction reported that the student
guide and troubleshooting guide worked effectively.
The circumstances under which student pilot course participants were identified and information
communicated back to the Project Team were not ideal and this process needs to be more
streamlined post-project. Learning Technologists require all online student contact details 5 days in
advance of a course to set up their Moodle/BBB accounts and schedule student induction into their
otherwise busy timetables of on-campus student/class support.
Within the SWEET Project, the Learning Technologists were tasked with the responsibility of setting
up student Moodle/BBB accounts and inducting the students prior to course sessions. Project Team
members visited online student locations on the first instance of joining an online BBB session to
assist with troubleshooting if required. This level of outreach support enabled the Project to identify
and resolve connectivity issues, informing the evaluation report and providing a vital source of
knowledge/experience which will inform further course development. The support was also ensured
that, in this early phase of system testing/evaluation, the participating remote students were not left
facing local difficulties in isolation.
Such levels of support, however, while crucial in the initial system testing phase, are not sustainable
long-term. Colleges cannot impose control over remote centres which they do not control directly as
part are not part of their institutional structure. Online student participation from remote centres
outwith direct college control will therefore always be subject to compatibility of local hardware,
software and internet connectivity.
Significant time was lost in the first 2 pilot course sessions while difficulties with remote student
connectivity and system performance were resolved. By the end of the 3 rd pilot course these
problems had been largely resolved. One of the tutors commented that the knowledge of connectivity
issues learned from the pilot will enable them to deliver an achievable set of expectations to the future
remote and in house learners.
38
Solutions
1) Students require to be fully enrolled prior to the session so they can have their own username
and password. An online student enrolment and induction policy, supported by a clear online
student enrolment process should be established and implemented.
2) The College should consider establishing partnerships with centres which can either directly
provide regular students or provide access to students within the area and guaranteeing
conformance with minimum BBB connectivity standards with initial student support if required.
3) An effective and responsive on-campus remote student Helpdesk support for partner remote
centres and other remote students should minimise the requirement for direct outreach
technical/induction support.
E4. What checks were made on student awareness and understanding of the new course delivery
methodology prior to starting the REHIS course proper?
Students were sent relevant information and shown online how to use.
When connectivity and system performance difficulties were experienced at the start of initial
sessions, student induction tended to suffer as tutors would typically concentrate on making up for lost
time.
Solutions
1) Tutor guidelines for mixed classroom/remote student course delivery should include clear
references to effective induction and course delivery practice, supported by checklists
covering different aspects on how the session will work and be made to work well, for
example:
that the tutor may summarise classroom discussion;
when looking for an answer from a remote learners address them directly by name;
Effective use of Powerpoint
Use of desktop sharing to access video and other formats of learning materials not
supported by the BBB whiteboard.
Student discussion around who has used Skype, Xbox, instant messaging, etc, in order
to ascertain familiarity and link the features of BBB to known systems.
E5. What evidence of student responsiveness to induction can be gleaned from the recording of the
pilot course and focus group feedback?
None of the first and second pilot course classroom participants felt they had been particularly
prepared, but this was not a concern for the first cohort.
A couple in the second cohort had been given the wrong dates and then found out the correct dates
only by chance.
One participant also mentioned that shed been told the course would run over 2 weeks, but in fact it
was 3. [This last comment relates to Tutors having responded to course evaluation of the first course
which led them to restructure the course in order to effect improvements to delivery.]
39
The third cohort of classroom students were recruited to the course at the last minute and received
little or no information about the course. There was a general view that it would be helpful to receive
an outline of the syllabus on one side of A4 prior to the course.
Solutions
1) In the event, recruitment of students to the pilot courses was very last-minute and reflects
course management practice which should be improved post-project. Clear and effective
communication with students, both pre-course and through induction, applies to any course
irrespective of delivery mode. College policy and procedure in this respect simply requires to
be applied as BBB delivery is rolled out.
F. STUDENT EXPERIENCE
F1. In what way(s) the new course delivery methodology impact on the classroom cohort experience
compared to previous classroom-based student cohorts?
Some appear to have enjoyed it and others were disappointed in the lack of direct communication
with online participants.
F2. In what specific ways did the new course delivery methodology improve and/or reduce the quality
of the student experience on-course?
The following comments provide a tutors perspective:
Apart from the delays with the initial pilots, I dont necessarily think that the quality of the experience
was improved or reduced, the benefits to the remote learners were really about easy access to the
learning, reduced travel issues etc.
The following comments provide a perspective from the Learning Technologist who worked directly
with remote students:
Students commented that they would have liked to try the equipment that showed the bacteria on
your hands. It was not easy to see remotely it working as there was light above that showed lines
and camera shaking. Suggest that if this is used again that something be devised to cover the
aperture at the top perhaps with a hole for the webcam as this was shaking. Otherwise the students
enjoyed the experience. They were fully engaged in the sessions.
F3. What can we learn from evidence of student reactions, body language, interaction and comments
during the course itself. Use the recordings made of the course as evidence.
Again, outwith the initial problems relating to connectivity, I was not aware of any issues apart from
the disappointment at the lack or limited interaction possible between the two groups.
Some students did not like seeing themselves on screen. I advised them to minimise but to make
sure they were sitting so the class could see them. Being in attendance at a few of those sessions
the students were happy and engaged.
The following comments provide an observers perspective of the remote student participation in the
first pilot course:
40
During the 3 hour sessions the students look disinterested, especially when they were not taking part
in any of the questions. They looked more lively during sessions when they were being asked for their
opinions and commenting back verbally or through chat.
F4. How were online and classroom students able to communicate with each other during the course
and how would you rate the quality of student-student communication (including classroom cohort
-classroom cohort, classroom cohort online and online-online)? What improvements, if any, could
be made?
On only one occasion were we able to put the remote learners audio through the smartboard; at all
other times controlling the feedback proved more than was possible. I think that the presenters
relayed the communications between the two groups as much as was possible, but this was generally
disappointing due to the inability to provide direct communications between the two groups.
The students were so busy listening they did not use the chat much unless they were asked to by the
Lecturer. When asked to respond the students acted immediately.
F5. What does the qualitative evidence tell us about the student experience and what changes need
to be taken to address issues raised by students?
When the system works well the evidence generally points to a reasonable level of user satisfaction.
G. ANY OTHER ISSUES ARISING FROM THE PILOT COURSE
The scale of the REHIS pilot course has not enabled the Project Team to stress-test BigBlueButton to
the point of unacceptable performance or complete system failure. The College needs to arrange
sessions which will establish the optimum and maximum number of simultaneous participants within a
single course session and the maximum number of simultaneous course sessions which can be run at
any one time. This will inform central scheduling of BigBlueButton sesions as the system is deployed
across the College.
G1. What other issues have emerged from the pilot course, both positive and negative?
From a tutors perspective:
The level of teamwork between Technical staff and non-technical staff has improved dramatically over
the pilot. Had that team approach been more evident earlier in the pilot issues may well have been
sorted out much earlier.
From a Learning Technologists perspective:
The view of the classroom instead of the Lecturer should be considered for future classes, e.g. from
the student view showing the Lecturer and the screen. All questions raised by the students in the
class would be better if they could be heard by the remote learner or the Lecturer to convey this to
them this was getting better as the trial went on.
Scheduling of Learning Technologist time required to provide outreach support to remote learning
centres became an issue where project requirements came into conflict with other timetabled
Learning Technologist activities.
41
College laptops are not currently configured to use BBB in remote locations or on-campus.
G2. How do these now need to be addressed within the project and in preparing for potential rollout of
BBB post-project?
The BBB audio issues need to be resolved as a priority.
Expert technical support needs to be extended in order to provide sufficient access to course
development and delivery activities when required.
If Learning Technologists (Learning Technologists) are to continue to be involved sufficient time needs
to be booked in their timetables.
More information needs to be supplied e.g. Learner information in plenty time to arrange all required.
New equipment will need to be purchased,( e.g. webcam with no sound on it and better
headphone/mics) with a budget allocated for this investment.
College Laptops need to be configured to use BBB in class.
G3. Tutor/LA reaction to technical issues encountered during live sessions with students
1. When problems occur, tutors shouldnt ignore classroom cohort or turn backs on students for
significant periods of time.
2. The Project team discussed how the presenter should stand as we saw a lot of their backs. It
was felt that the webcam should be in back of the classroom facing into the Smartboard and
presenter.
3. Negative comments about system in front of students can undermine student confidence in
course/College and should be avoided.
4. A contingency plan is needed for working with both classroom cohort & online cohort when
technical issues occur. Also, what would happen if fire alarm goes off?
5. Possible solution to loss of online student connectivity: record tutors doing presentation &
prepare set of questions for students to consider and answer from own experience ask
students to feedback some points via online chat refer online students to presentation if link
lost & arrange online tutorial to discuss issues.
42
APPENDIX E: REHIS STUDENT COURSE EVALUATION SUMMARY REPORT

SWEET PROJECT
BigBlueButton REHIS Student Focus Group Findings
The focus group facilitators met with course participants on two occasions:
Group 1 a) face-to-face discussion with 3 classroom-based learners on Monday 10 October

2011, 5.15-6.15pm;
Group1 b) telephone discussion with one distance learner on Wednesday 12 October 2011,
1010.30am. It was not possible to establish a telephone discussion with the second
distance learner.
Group2 a) face-to-face discussion with 7 classroom-based learners on Monday 21 November
2011, 4.30-5.30pm;
Group 2 b) telephone discussion with 5 distance learners in the course of that week. It was
not possible to establish a telephone discussion with 2 of the distance learners.
Group3 a) face-to-face discussion with 8 classroom-based learners on Monday 28 November
2011, 12.30-13.30pm;
It was not possible to establish a telephone discussion with the 2 Group 3 distance learners.
The purpose of the discussions was to gather student feedback on their experience of mixed
classroom/online learning in order to help evaluate the course. The aim was to establish what worked
well and what worked less well for participants.
Feedback was sought on the following specific issues:
1. Previous positive/negative learning experiences (with or without use of technology)
There were mixed responses within the discussion of previous learning experiences:
Only 1 of 12 Group 2 member expressed positive experience of previous learning and this was
predominantly due to having enthusiastic, motivated teachers. He had also enjoyed the social side of
being at school and college.
The other 11 Group 2 members had found school uninspiring and dull due to uninterested teachers,
poor classroom management and there being no apparent real-life relevance to lessons.
It was felt by one participant that learning derives from the learners own attitude and motivation. She
felt that she would have been more motivated at school if technology had existed at that time. She
finds the use of technology in the classroom exciting, believing that access to computers would have
allowed her to participate more in her education and thus to feel more engaged in the process by
being able to do things for herself.
One slightly older participant felt that although she had enjoyed learning at school, she could have
achieved more had technology been available. She specifically mentioned that the use of computers
with Spellcheck would have been a huge benefit to her and she also felt that learning for her might
have been more fun with technology: film clips, images, etc.
Another older student referred to new technologies as simply part of 21st century living which people
have to keep abreast of. When she was at college 30 years ago, chalk & talk was the predominant
approach. Returning to study 20 years later she found that individual learning, using new
technologies and other learning resources was the norm, which she viewed as a positive
development and more positive experience. Now, technologies like BBB are introducing further
changes which, technical glitches aside, she views as continuous improvement.
A younger learner had used technology in the school classroom but had only found it distracting. It
did not enhance her learning experience.
The youngest participant was least positive about technology in learning. She had completed an
43
HNC course at college where We just sat in front of a computer screen the whole time. When the
lecturer went out the room we just mucked about and when she came back to see how we were
getting on all we had to do was press a button and look as if we were working on something. It was
very demotivating.
One student expressed a preference for smaller face to face classes with more interaction between
tutors and students.
There was general agreement in Group 3 that online learning was more suited to shorter courses.
2. Use of social networking applications/websites
Most classroom participants actively use Facebook and Skype in particular and were very animated
when discussing its benefits, mainly to keep in touch with distant friends and relatives.
As a single mum at home a lot, it gives me a sense of still belonging by seeing what everyones up
to. I feel like Ive still got some kind of social life.
It changed my life because I found my adopted daughter after all these years.
I have been back in touch with lots of distant relatives. They even came over for my wedding.
I couldnt do without it now.
The oldest Group3 student felt interaction and communication via Facebook was very shallow.
Asked how BigBlueButton compares to Facebook, the 4 Group 1 participants were very very
enthusiastic. They compared BBB to Skype and all agreed that because it allows people flexibility in
their learning, it is a great innovation. Discussion focused on the benefits of BBB to, for example,
people who want to learn but are restricted by disability or illness and people who are natural loners or
shy and would not want to come to a physical college space but would still like to learn. All
participants thought that BBB would only encourage learning. The Group 2 participants generally felt
it was quite good but still had too many glitches and therefore slowed down the learning process.
One online learner said that she had never used Skype before and felt strongly that if she had, she
would have managed this online experience with more ease. She would have liked to have been
offered a half hour session introducing her to the technology before getting thrown in at the deep
end. She felt her learning would have been better if shed had some adapting time to the procedures
as shed been overwhelmed by the technology in the first lesson.
Another online learner who has previously used conference calling, found BigBlueButton very
frustrating and much less sophisticated than her previous experience.
Another online learner felt he had coped well with the experience because he was very IT literate. He
felt much of the success (or otherwise) depended on a participants existing IT skills, particularly
since so much quick typing was involved when there were sound problems.
3. Preparation received prior to the course
None of the Group 1 participants felt they had been particularly prepared, but this was not a concern.
None of the Group 2 participants felt they had been particularly prepared for the course. A couple had
been given the wrong dates and then found out the correct dates only by chance. One participant
also mentioned that shed been told the course would run over 2 weeks, but in fact it was 3.
Group 3 classroom students were recruited to the course at the last minute and received little or no
information about the course. There was a general view that it would be helpful to receive an outline
of the syllabus on one side of A4 prior to the course.
4. Expectations of the course
44
The three Group 1 classroom participants agreed that the course was more demanding than they had
expected. They had each believed that the course content would be mostly common sense and were
surprised it was at such a high level.
The Group 1 online learner found that the course met his prior expectations.
All Group 1 participants found the tutor delivery great, friendly, funny and not at all intimidating.
All Group 2 participants found that their expectations of the course were either met or exceeded.
The Group 3 classroom students appeared to have few prior expectations of the course.
One student knew that the course assessment would be multiple choice. Another student expected
the course to take a common sense approach where the focus would not be on food production but
rather on everyday food handling issues.
5. Balance between theory and practice
Again, the three classroom participants were in agreement in that they would have preferred more
practice on the course, whereas the online learner felt that the balance of theory and practice worked
well for him.
The classroom members had much discussion about this and agreed that half theory-half practice
would have suited them better. They said they would have been happy to bring along materials if a
large work surface could have been brought to the classroom. One student felt that it would have
been better if they could have spent some time in a college kitchen. This would have improved her
learning.
All participants felt that there was a good balance between theory and practice, although there was
some agreement that having a video of a real kitchen being used could have enhanced the learning.
Classroom students felt the balance between theory and practice was good. The video of the kitchen
was particularly useful and welcomed by the group, as was the handwashing exercise. The group
recognised, however, that the online students did not benefit from the handwashing exercise as they
were not physically present in the room.
6. Balance between tutor presentation, student discussion and student activities
Group 1 students reported no student activities. Tutor presentation was excellent and student
discussion was encouraged. However, while this worked well for the learners in the classroom, it was
less satisfactory for the online learner as he could not hear what they were saying. While he said that
this did not bother him, the learners in the classroom were adamant that the online learners were not
receiving as good an experience as they did, due to not being able to participate directly in discussion.
There was abundant praise for the tutors from all Group 2 participants! Their styles complemented
each other very well; they were interesting, funny and competent. They encouraged student
discussion and made good attempts to even translate classroom jokes to the online learners.
Generally, Group 3 classroom students felt that the balance was OK and knew that if they wanted to
know something they had only to ask. While the course did not involve a lot of discussion, classroom
students very much welcomed the opportunities to ask questions presented by the tutors throughout
their delivery. Tutors accounted well for learning differences among the class and made sure
everybody understood the topics covered. Students referred to the tutors as very relaxed and friendly
and well able to put the students at ease.
7. Range of learning materials/resources
45
Group 1 students found the books and worksheet useful and they enabled them to revise. The online
learner did not receive the resources the first week as they were sent to the wrong address. Once
received, he found they contained all he needed for the course.
Group 2 participants found the books and worksheet useful but there was a general consensus
(classroom and online) that also having materials online would be beneficial. One online participant
felt that it would be useful to record the lessons and have them online for students to review and
revise from. Another participant felt that having Ron and Nan as resources was far better than the
contents of the book; their knowledge and information was more relevant.
Group 3 students appreciated being able to take information away with them and having to answer
questions at home to reinforce their learning. The recap at the end and general revision of earlier
sessions was considered to be very useful.
8. Course assessment arrangements
Responses on assessment arrangements varied.
The students did know that the assessment would take the form of multiple choice questions,
although none of them knew the length of the assessment. All 3 Group 1 classroom participants felt
baffled that some of the questions in the assessment felt as if they had nothing to do with what wed
just been learning. In this sense, they felt a little under-prepared.
Some Group 2 classroom participants felt that they were fully prepared for the assessment and liked
that there was a warm-up question and answer session just beforehand to consolidate learning.
One Group 2 classroom participant actually felt that there was too much exam preparation
considering the level of the course and that they were taught more than they required to pass the
assessment.
None of the Group 2 online participants said they had been prepared for the assessment. None knew
how long the assessment would last or what format it would take.
The Group 3 classroom students found the course assessment arrangements straightforward, with
enough time to answer the questions.
Given the uses of new technology in course delivery, one Group 3 classroom student expected the
assessment to be online rather than paper-based. The students were aware that the online students
were not completing their examination until next week and felt that these students should also benefit
from a recap session beforehand.
The Group 3 students found the mock questions in the course booklet useful but indicated that more
such questions might be useful.
9. Most useful/least useful parts of the course
All Group 1 participants found the course extremely useful. All commented on its direct relevance to
their life at home and at work. The online participant said that the course had contributed to him
changing some of his work practices. The classroom participants felt that sharing their experiences
and being able to talk things through with each other and the tutor was very beneficial.
The least useful part of the course was felt to be the time spent by Learning Technologists attempting
to get the technology to work. BBB kept crashing in the first week and when it froze the online
participant could not see or hear anything. He was communicated with by telephone.
The video was not useful to the online learner as he could not see it.
One classroom learner would have liked the course to be spread out over a slightly longer period with
more practice activity incorporated.
46
All Group 2 participants found the course useful and commented on its direct relevance to their life at
home and at work.
The Group 2 classroom participants felt that sharing their experiences and being able to talk things
through with each other and the tutors face to face was very beneficial. They felt that the online
learners missed out in this respect.
One Group 2 online participant suggested that holding the course in a local Community Centre or
library would offer not only faster connection speeds but also someone experienced on hand to help
with technical difficulties or to help participants lacking computer skills.
The Group 3 classroom students found all parts of the course relevant. One student felt that the most
useful part of the course was understanding that you were the main risk when working in a food
production/distribution environment. Personal hygiene, raw food risk factors and microorganisms/pathogens were individually highlighted as particularly useful.
The least useful part of the course was felt to be the time spent attempting to get the technology to
work. BBB did not work in the first week for 20 minutes (kept logging online participants out) and,
although the classroom participants were able to talk course content through with Nan, the online
learners expressed some frustration at having to wait and then to experience echo from the webcams
and no sound.
10. Advantages/disadvantages of taking the course online and in the classroom.
Preferences?
All Group 1 and Group 2 participants agreed that the major advantage of taking the course online is
that it provides the learner with flexibility and choice. All participants agreed that the major advantage
for online learners was that they did not have to travel or take time off work to attend college. Had
she been able to do the course online, one of the classroom participants said it would have saved her
3hrs travel time, having had to take 3 buses to get to college.
All 3 Groups also agreed that the disadvantages of taking the course online were the technical
problems and wasted time. Having said this, however, the students said that as they knew the
course was a pilot they were more tolerant of the interruptions.
The fact that the distance learners could not see or hear much of what was going on was another
perceived disadvantage. The Group 3 classroom cohort could see the online learners on-screen but
could not communicate directly with them, which they would have preferred to do. This seemed to
trouble the Group 1 online learner less than the Group 1 classroom learners who strongly felt that he
was missing out by not really being part of the class. One Group 2 online participant said the biggest
disadvantage for her was that when there was no sound, she had to type in her responses. She was
embarrassed that everyone could see these and she said it made her feel very vulnerable.
The 3 classroom learners unanimously preferred to be in the class rather than online:
It got me out of the house
I got to meet new faces
I was able to share ideas
I got some answers from the other people in the group
There would be too many interruptions at home or at work
Having said that, they all liked the choice that BBB can offer.
All in all, however, all online learners would choose to do the course online again in future if given the
choice.
47
The stated preferences by classroom and online students for their respective modes of delivery
should be interpreted with some caution, however. Research evidence demonstrates that given 2
choices, groups who directly experience one option will tend to express a preference for that option
There was general agreement that improved access to training/learning offered by BBB was an
advantage for remote learners.
The oldest student commented that she wouldnt want a camera focussed on her as a student for so
long, although admitted that this was more of an initial impression during the first session which
diminished as time went by.
11. Tutor attention
Again, all participants highly praised the tutors for being interesting, inviting questions, using good
humour and not being intimidating. The online learner had the tutors there to convey to him what
classroom members were saying and appreciated the effort made by tutors to translate to them what
classroom members were saying.
Classroom students did not feel disadvantaged as a class in having the remote learners participate
simultaneously but admitted that it very much helped having 2 tutors present at the same time,
enabling one to deal with the group while the other sorted out technical issues.
Classroom students referred to instances where the tutors occasionally became confused about what
they had covered already, mixing up their group with one of the other cohorts. Also, tutors did not
explain the reasons for the filming of the session for the project.
12. Communication between classroom and online students
Communication between classroom and online students was minimal due to the limitations of the
technology. While the online learner did not feel disturbed by this lack of direct communication (due
to being unable to see or hear much of the interaction), the classroom learners felt they would have
liked to communicate more with those online so that they felt like one whole group on the course. It
felt like there were two groups: an online group and a class group. One classroom learner felt that
the lack of communication meant that the learning lost its flow.
While the Group 2 classroom participants felt that the online learners missed out on the camaraderie
of the classroom experience, the online participants said that this did not bother them at all!
The Group 3 classroom students recommended setting up 2 cameras in the room to enable the online
students to see the group.
The Group 2 online students could use the chat function within BBB to answer questions but their
logon setup only displayed each student chat message as Guest which meant that the classroom
group did not know who was posting questions/comments.
Overall, while aware that they were suggesting more communication, the Group 3 classroom students
recognised that this may not necessarily add any value to their learning and may be worth less effort
than it might take to redesign course delivery in order to improve direct communication.
13. Learning Technologists
The presence of the Learning Technologists did not benefit the classroom learners and they felt it was
unhelpful to have lost an hour while the Learning Technologists focused on technical problems. They
felt they just had to be patient and wait for the class to start the first week. However, it was
acknowledged by all Group 1 participants that absence of Learning Technologists would have meant
that the online participants could not have taken part.
The Learning Technologists were extremely helpful to the Group 2 online learners.
48
The Group 3 classroom group had no views on Learning Technologists as they were deployed to
support the online students and tutors in using the system rather than the classroom cohort.
14. Further training
All Group 1 participants felt that this course had stimulated them to continue their education/training.
They each agreed how much they had enjoyed the learning.
The Group 1 classroom participants discussed how much they liked the buzz of coming into a college
full of students and that they had gained confidence from this experience. They agreed that they
enjoyed keeping their minds active. None, however, was keen to undertake a full-time course but
would like to continue to learn for a few hours a week.
Possible future courses included: Accounting & Bookkeeping, Beauty Therapy and Catering. The
online participant was keen to undergo some more work-related training and would be happy to do
this online.
The main proviso discussed by the Group 1 classroom participants around future learning was the
reality that college qualifications did not guarantee jobs and that even with the confidence gained in
training, being rejected for jobs would quickly knock this back.
4 Group 2 participants said that this learning experience might encourage them to explore further
college courses: English, Motor Maintenance, Engineering and Veterinary Nursing.
The Group 3 classroom students expressed no interest in pursuing further training after completing
the REHIS course.
Jan Crawford / Donald Steele
5 December 2011
49
APPENDIX F: SYNCHRONOUS COLLABORATIVE LEARNING CASE STUDY
Synchronous Collaborative Learning case study 1: REHIS

Elementary Food Hygiene at Jewel & Esk College
The SWEET Project
The SWEET Project received a years development funding from the JISC SWaNI-FE Programme
over the period January-December 201. The Project aimed to:
Establish and evaluate an open source educational web-conferencing system (BigBlueButton)

within Jewel & Esk College;
Develop and evaluate a synchronous collaborative learning (SCL) model, delivering training
simultaneously to a group of classroom based trainees and employees in the workplace, with
the remote students participating via web-conferencing;
Apply the SCL model to an existing training programme in order to evaluate the methodology
and report the results to the wider education and training community.
This case study outlines the methodology, reports key findings and provides links to further
information and resources available on the SWEET website.
The pilot course: REHIS Elementary Food Hygiene
The REHIS Elementary Food Hygiene course was selected to pilot the SCL methodology as it is short
and practical, making it suitable for redevelopment into a package of learning chunks which students
would be able to complete in short structured periods of learning.
This would enable completion of the course to be built around the employees work patterns and
commitments. Most importantly, achievement of the REHIS qualification itself satisfies a legal
requirement for those undergoing the training, providing a strong motivator for employers and
employees to complete the training.
Through its experience of working in the food and hospitality sector, Jewel & Esk College had
identified specific operational difficulties for employees with potential to be addressed through
synchronous collaborative learning and BigBlueButton. Relatively low numbers of employees in
individual work locations allied to tight staffing and shift patterns means staff may only be available for
very short periods of time with no time available for travel. Training can be delivered face to face on
the employers premises in short sessions but the travel, tutor time and set up costs incurred by the
College and passed on to the employer increases where a trainer needs to go out on multiple
occasions. Thereafter, where new employees require to be trained individually as they join the
business, on-site training becomes less viable.
The opportunity to participate in larger campus-based training sessions via web-conferencing was
expected to enhance the remote online employees training experience through interaction and
collaborative learning with other students. The project therefore provided a suitable operational
environment and evidence base from which to carry out a cost/benefit analysis of synchronous
collaborative learning.
Traditional course delivery arrangements
The standard course is delivered face to face with learners over a single day over 6 hours. The 6
hour period can be tailored if required, with delivery in 3x 2 hour or 2x3 hour sessions
The tutor is responsible for:
Collecting handbooks, examination and course materials from REHIS;
Photocopying pre-examination revision questions for students;
50
Checking the REHIS power point presentation;
Booking a kitchen for students to visit in order to review the working environment;
Arranging overalls for the kitchen visit.
Synchronous collaborative learning delivery arrangements

Course delivery was structured into 3X2 hour sessions over a 3 week period, primarily to break up the
learning for the online students into more digestible chunks. This approach worked to the benefit of
all learners, however, as it:
Enabled them to consider the topics covered within their own working environment; and
Allowed time for the learners to complete revision exercises and read the REHIS course
handbook between each session.
In the event, therefore, adapting the delivery pattern to meet the needs of online learners provided all
learners with the opportunity to reinforce their learning during the course in a way which was
impossible within a single days delivery.
The Colleges team of Learning Technologists contacted the online learners prior to the course in
order to check that they could log on properly and understood how to use the system.
The course tutors:
Redesigned the delivery sessions to cover the syllabus more efficiently, thereby providing
additional time for dealing with online participants.
Re-aligned the standard REHIS course Powerpoint presentation to the new course design,
thereby ensuring that that the delivery flowed well with the course syllabus.
Organised a video clip of the working kitchen environment to replace of usual organised tour
in order that both online and classroom-based learners could simultaneously view the correct
surfaces and practices in an appropriate working environment. This also freed up additional
course time required to support SCL delivery.
Used delivery time saved and student activities set between sessions to re-cap on the
previous learning and to go over the revision questions with both the face to face learners and
the on line learners together. This worked well as online learners could submit questions and
answers via audio and online chat.
Evaluated each session with the students through discussion and a structured student
questionnaire.
Worked with the Learning Technologists to develop a secure REHIS on-line multiple choice eassessment which would enable online students to complete their examination remotely. The
invigilation required for remote assessment would be enabled through direct web-cam
supervision by the tutor via BigBlueButton.
Main changes in SCL delivery.

The main changes between the delivery of the standard hygiene course and the SCL/BigBlueButton
course relate to tutor presentation, timing and access to online resources.
Tutor as conduit
The presenter at times must act as the link between the in-house and remote learners in relaying
some of the in-house interaction to the remote learners and vice versa. The amount of time spent
51
relaying information between the two groups depends on access to equipment (eg smartboard, open
room microphone) which enables students to communicate with each other directly. The tutors found
that access to such facilities would be the preferred option as this would:
Substantially reduce or completely eliminate the time spent acting as intermediary; and
Improve the quality of communication among all participants.
Within the pilot project, the audio quality of online students when directly relayed via smartboard
speakers was not sufficient to make this feasible but subsequent investigation indicates that this can
be resolved with some additional expert technical support in adjusting audio settings.
Timing
The addition of remote online students requires additional preparation time by the tutor who, in the
role of BigBlueButton presenter, must:
Check and ensure connectivity well ahead of the start time; and
Ensure any electronic course resources are in place ready to be uploaded at suitable slots
throughout the planned delivery session.
Presenter skills
The tutors had to develop a presentational style which was suited to simultaneous direct
communication with remote and classroom-based students. As presenters the tutors also had to
develop sufficient knowledge of and competence in using BigBlueButton functions in order to
anticipate and quickly rectify common technical issues as they arose.
The SWEET Project was evaluating a technology and delivery methodology with no institutional
precedent. Consequently, the tutors had no peer group experience on which to draw in preparing
themselves for actual course delivery. During the first pilot course tutors encountered significant
interruptions arising from technical problems in both the classroom and remote student locations
which they were unable to resolve independently.
By the third pilot course, however, they had developed sufficient technical knowledge through direct
experience which enabled them to:
Anticipate student connectivity and system performance issues; and
Either eliminate them entirely or apply immediate solutions when encountered.
The pilot course tutors are now in a position to share their new knowledge and skills with colleagues
who will now enjoy the benefits of peer support not available to the SCL trailblazers.
Online resources
The BigBlueButton web conferencing system was selected because it:
Is available as open source software and therefore incurs no software licence costs; and
Can be plugged directly into and used alongside Moodle, the open source virtual learning
environment (VLE) system with the largest user group in the Scottish tertiary education
community.
The SCL delivery methodology exposed the REHIS pilot course tutors to the opportunities presented
by VLE-supported course delivery, leading them to develop new online materials for all students to
use independently between course sessions. The provision through Moodle of on-line notes, course
handbooks and online quizzes greatly improved the course for the learners as, by reading notes and
completing parts of the quiz, they quickly become more familiar with and competent in the
knowledge/skills to be learned and the material to be covered in their handbooks.
The learners completed online Moodle quizzes between course sessions while tutors used them as
the base for recapping and moving on through the course in their presenter role. The same quizzes
52
also provided a means of formative assessment which enabled both students and tutors to identify
areas requiring remediation or additional input in preparation for the summative assessment exam.
Student experience and feedback
The pilot course was delivered simultaneously to 3 cohorts of classroom-based and online learners,
comprising 18 classroom-based and 8 online students in total. Students provided feedback in
discussions with tutors, via a student evaluation questionnaire and within independently facilitated
focus groups immediately following the last course session.
As students were aware that they were participating in a pilot course designed to test new technology
and a new course delivery methodology, they were prepared to tolerate technical problems where
they arose. Were they enrolled as paying students on an advertised College course, however, the
students very clear indicated that they would not accept interruptions arising from connectivity and
system performance issues. By the third cohort, however, the majority of these issues were known
and resolved.
The major unresolved issue remained direct communication between classroom-based and online
students. All classroom students felt that the online students missed out on the benefits of direct
communication with the classroom cohort and wanted the opportunity themselves to speak directly
with the online cohort.
Three learners participated on both face to face and remote sessions. All three intimated after their
first remote session that they preferred this mode of delivery as they did not have to travel in to the
centre. Having attended a classroom-based session, however, they subsequently stated a preference
for the direct interaction afforded by face to face delivery. Within the pilot course online students could
not directly interact with the classroom cohort. This is an issue which the College will address postproject we are assured that a technical solution can be implemented.
Putting aside technical issues which are almost inevitable in a pilot project of this type, the students
were extremely positive about the tutor input and course delivery overall. Of the 25 students who
completed the final examination, 24 (96%) passed. The student who failed had particular learning
difficulties and, despite the provision of additional specialist support in the classroom, was unable to
meet the demands of assessment irrespective of the course delivery itself.
Further information
This case study provides a highly summarised account of the SCL/BBB course delivery model and its
evaluation. The full SWEET Project final report and a range of other guides, support materials and
evaluation documents are available on the SWEET Project website.
For further information contact:
Ron McGilp
Service Industries
Jewel & Esk College
Edinburgh Campus
24 Milton Road East
EDINBURGH
EH15 2PP
Tel 0131-344-7325
Email: RMcGilp@jec.ac.uk
53
Synchronous Collaborative Learning: course planning, setup and delivery model
APPENDIX G:
SYNCHRONOUS COLLABORATIVE LEARNING: COURSE PLANNING, SETUP AND DELIVERY MODEL
What is synchronous collaborative learning?

Synchronous collaborative learning (SCL) is a course delivery methodology whereby learners in off-campus locations join
scheduled on-campus classes via web-conferencing connections, with on-campus and off-campus students simultaneously
receiving tuition, interacting and participating in group learning and assessment activities. The methodology removes
barriers to learning created by:
Geographical distance;
Requirement to travel to campus locations; and
Release from employment.
While originally developed to enable off-campus learners to participate in on-campus classes and other group learning
activities, multi-campus educational institutions can also use the course delivery methodology to bring together students
based in different campus locations for shared learning activities. SCL can also be used to include students from different
educational institutions into group learning activities.
SCL a model approach
The synchronous collaborative learning methodology applies a standard approach with common key elements required for
the course delivery to work. The detail of these elements, however, will vary from institution to institution depending on
local access to facilities, ICT and other support services.
Taken together the common key elements represent a standard course planning, setup and delivery model for synchronous
collaborative learning.
This document presents a generic model for SCL which seeks to identify specific requirements, subdivided into key elements
which are categorised as essential or desirable for SCL course delivery and similarly categorised for off-campus participation.
Key considerations relating to course planning, support and delivery are identified and briefly outlined/reviewed under each
key element within the overall SCL model.
54
The SWEET Project Synchronous Web-Enabled Employee Training

The SWEET Project received a years development funding from the JISC SWaNI-FE Programme over the period JanuaryDecember 2011. The Project aimed to:
Establish and evaluate an open source educational web-conferencing system (BigBlueButton) within Jewel & Esk
College;
Develop and evaluate a synchronous collaborative learning (SCL) model, delivering training simultaneously to a group
of classroom based trainees and employees in the workplace, with the remote students participating via webconferencing;
Apply the SCL model to an existing training programme in order to evaluate the methodology and report the results to
the wider education and training community.
The SCL model presented here draws from the experience of the SWEET Project participants. These included tutors, Learning
Technologists, expert technical support staff and, last but not least, the students who participated in the pilot REHIS
Elementary Food Hygiene course used to develop, pilot and evaluate the SCL delivery model. We describe the key elements
of synchronous collaborative learning as a generic model, capable of being applied and adapted to any learning programme
involving simultaneous participation by on-campus groups of learners and off-campus online participants.
The SWEET Project website provides a full Project report, together with other documents, guides and evaluation tools which
may be of interest to educational institutions and course tutors considering adoption of the model.
Donald Steele
SWEET Project Manager
12 January 2012
55
Synchronous Collaborative Learning model key elements and key considerations

Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
Internet
access
Robust institutional
ICT network.
Essential
Essential
ICT network must be reliable and operational at all scheduled SCL

delivery times, otherwise capacity of institution to support and
deliver SCL courses is compromised.
Fast broadband link
Essential
Very strongly
recommende
d
The delivery institution requires a fast broadband link in order to

ensure the highest quality audio/video transmission to and from
learners and match the capacity of the best broadband connections
available to off-campus participants.
Essential for
video usage
Webconferencing
application
Experience shows that, while slow internet connection speeds at

remote student locations can enable a working link to
BigBlueButton, this does cause significant connectivity and
performance problems, particularly with respect to webcam use and
video presentation transfer to the student PCs.
Web browser
Essential
Essential
All communication between the SCL tutor and remote students and
classroom-based students is web-enabled. Individual webconferencing systems may be more or less reliable when used with
particular web browsers. This needs to be established from the
outset of SCL delivery and fully compatible web-browser(s) installed
on all campus computers. Preferred browser information needs to
be included in the local computer specification issued to online
participants.
Dedicated server
Desirable
Not required
Where the educational institution can host the web-conferencing

system on a dedicated server this is the ideal setup. Where only
limited numbers of users are online at any one time, a shared server
may suffice.
Commercial or open
source web-
Essential
Not required
A range of web-conferencing applications are now available from

commercial and open source providers. Functions common to most
56
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
conferencing
application
Key considerations
are:
audio/webcam transmission;
electronic blackboard;
presentation functions;
ability to change presenter;
icon for drawing presenters attention;
online chat;
desktop sharing.
Commercial systems tend to have additional refinements and

functions.
While open source systems incur no software licence costs, there
may be costs to the institution arising from additional technical
support and development of the application.
Compatible
institutional network
security policy /
setup
Essential
Essential
Some web-conferencing applications require external participant

computers to download and install computer code which enables
participants to fully engage in course sessions. The SCL delivery
institution must include this factor as a system procurement
criterion when choosing a web-conferencing system because users
in workplace locations may be excluded by their network security
policy and setup.
Where the web conferencing system functions may be liable to
institutional blockage it is crucial that the educational institution
advises the student / employer of this prior to student enrolment
and makes reference to it in the external participant local ICT setup
specification.
Virtual
learning
environment
Commercial or open
source VLE system
Desirable
Not
applicable
Most educational institutions maintain their own virtual learning

environment as a resource for course delivery and student support.
57
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
SCL provision can be designed without any VLE usage but course
delivery can be significantly enhanced when web-conferencing and
VLE usage are brought together. For example, practical exercises or
demonstrations can be recorded and presented during SCL sessions.
Course resources and documentation can be uploaded to the VLE for
student use between sessions. Students can use the VLE to
communicate or work together between scheduled sessions.
Person to
person
communicatio
n equipment
Networked computer
Essential
Essential
The SCL delivery institution maintains and controls its own computer
hardware and software but exercises no control over the computer
workstations used by students in external locations.
It is crucial that a minimum specification for participants computers
is drawn up and implemented across the campus network. Failure
to do this limits the choice of locations from which to deliver SCL
courses and could interrupt or worse still completely halt delivery of
a live SCL session.
Institutions delivering SCL programmes have no control over remote
participant computers but must ensure, as required to join an SCL
course online , that external participants receive sufficient support
to:
Combined
earphone/microphon
e headsets
Essential
Essential
check their computer;

install required software; and
adjust settings
Audio communication is the principal form of communication

enabled by web-conferencing. Course tutors and external
participants use an earphone/microphone headset to talk to one
another during an SCL session.
SCL tutors will normally have to adapt their presentational style to
58
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
communicate simultaneously with a classroom based group and
online group of learners, ensuring that both groups receive equal
attention within a session.
Where tutors use a headset wired directly to a computer, this can
limit physical movement throughout a session. Wireless headsets
may be required where tutors demonstrate practical skills in which a
trailing headset cable may become an obstacle.
Speakers
Desirable
Not required
Speakers in the classroom enable remote learners to be heard

directly by the classroom cohort.
The presence of active speakers in the online students location,
however, can transmit serious audio feedback echo to all
participants.
During their induction, online participants must be instructed to
mute their speakers. Where necessary, the tutor may have to mute
the microphones of individual online learners whose use of speakers
impairs audio quality for other participants.
Web camera
Essential
Desirable
The use of a webcam by the tutor enables remote students to see

the presenter and, if required, view the classroom cohort or any live
practical demonstrations.
The resolution of the webcam and network bandwidth available to
remote students can impair the video quality, however, and as the
audio and video signals are processed at different speeds there is
typically some lack of synchronisation between the audio and video
transmission. Where more than 4-5 online participants join a single
session using a webcam video link this can significantly slow down
processing time on users computers, including the tutors.
The tutor therefore needs to consider system performance
59
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
implications of using webcams and recognise when their use adds
nothing positive or may even become detrimental to the student
experience and learning process.
Expert
technical
support
Network services
Essential
Desirable
The provision of readily accessible and responsive expert technical

support from the educational institution ICT/network services
support team is an essential requirement for SCL delivery. Tutors
plan the students experience of SCL delivery sessions on the
premise that unbroken and adequate performance levels of internet
connectivity will be guaranteed throughout a scheduled session.
Where connectivity or institutional computer workstation problems
interrupt delivery of an SCL session, both students and tutors
require these to be resolved quickly and efficiently. The required
level of expert support should be guaranteed within an institutional
service level agreement.
Where an adequate level of expert technical support cannot be
deployed as required to support SCL, then the introduction or
continuation of this delivery model should be postponed until
adequate support becomes available.
Computer
workstations
Essential
Desirable
Web conferencing systems are designed to function via established

web-browsers. Optimum performance and actual functionality,
however, can depend on the settings applied to and software
available on individual computer workstations available to users.
Educational institutions developing SCL provision must first ensure
that all institutional student and staff workstations conform to the
optimum performance specification for the web conferencing
system prior to full deployment.
Where particular workstation settings are required for web
conferencing usage the institution should consider setting and
60
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
locking these settings on all workstations used in SCL courses.
Where external participants access SCL provision from workplacebased PCs they should ideally have access to a local ICT support
service which can check and set up the workstations against the
specification provided to the learner .
SCL tutors, however, must recognise and plan for online
participation from students without access to local expert support,
with contingencies which allow for remediation of technical
problems when they arise and/or subsequent remediation of time
lost as a consequence.
General
technical
support
VLE/webconferencing systems
Essential
Not
applicable
The success of SCL programmes is dependent on reliable and fully

functional web-conferencing systems and, where used, VLEs. The
same general considerations concerning access to expert
institutional technical support outlined above apply here.
Learning
Assistants/Technologi
sts
Essential
Essential
In all but relatively exceptional circumstances, expert technical

support staff are not directly involved in assisting tutors and
learners to make the best use of computer applications and related
resources on a day to day basis. This type of service is more
typically delivered by computer-literate and learning support staff
employed as Learning Assistants, Learning Technologists, etc.
Course tutors, particularly at the initial stage of SCL planning,
development and delivery, require access to technical support from
staff directly involved in and employed to support student learning
and course development which involves use of new technologies.
This type of support requires practitioners who are competent ICT
users and possess a sound operational knowledge of the e-learning
61
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
and learning resource development and delivery applications
commonly used in learning resource centres. A key role of this
support service is the transfer of IT knowledge/ competences and
assistance in remediation of difficulties encountered by tutors and
students.
Where institutional deployment of web-conferencing and
development of SCL delivery patterns is being planned or rolled out,
senior managers responsible for this area of development must
understand the importance of providing tutors and learners with
access to technically competent support with learning support and
learning resource development experience.
This level of support is crucial in addressing common and readily
resolved technical issues experienced by tutors and students which,
if referred to the expert technical support staff would exceed their
capacity to respond in a timely and effective manner.
Experienced peer
support
Desirable
Not required
Resolution of common problems, development of based on

professional and operational experience and an appreciation of the
benefits of mutual support networks are all part of the peer support
which naturally develops among effective teams and supportive
institutions.
At the earliest stage of introducing SCL to the curriculum, however,
an educational institution lacks the previous experience crucial to
the provision of peer support for this activity among tutors. While
tutors can and do learn new skills without peer support, as a they
become experienced in SCL delivery the institution should consider
developing a peer support network to help embed and encourage
further development of the methodology across the institution.
Tutor
General computer
Essential
Essential
While computers have long been an essential operational tool in

62
Requirement
Key elements
development
programme
skills
SCL
course
delivery
Off-campus
participatio
n
Key considerations
educational institutions, many tutors and students still lack a basic
knowledge and understanding of standard computer functions and
settings which would enable them to troubleshoot common
problems as these arise.
Tutors delivering SCL programmes should possess or progress
towards a sufficient understanding of general computer operating
system and common application functions/settings. This builds
their technical competence and confidence to a level at which they
can not only use applications effectively but also use their
knowledge and experience to resolve everyday problems for
themselves as they occur.
SCL delivery requires tutors to be able to respond quickly and
effectively to everyday technical issues as they occur and
demonstrate an easy confidence/competence as computer users
which is translated to their overall performance as classroom and
online presenters.
Web-conferencing
user skills
Essential
Desirable
Common difficulties encountered with SCL delivery relate to

technical issues which can be readily avoided or resolved by
experienced users. Users need to be able to recognise and
diagnose common problems in order to deal with them as they
occur.
The tutor development programme should include practical
experience of using and troubleshooting the institutional web
conferencing system(s), with a requirement that tutors demonstrate
a minimum level of competency before running an SCL session for
the first time. If tutors lack a general confidence in either the
system itself or their capacity to use it this will show in their course
delivery style and impair their presentational performance overall.
Using a web-conferencing system to deliver web-conferencing
training to tutors would be an ideal method of developing the
practical knowledge and competence required to become a
63
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
confident system user.
SCL pedagogy
Essential
Not
applicable
The tutor must consider the different methods of learning accessible

simultaneously to a classroom group and online group of students
and build this into course activities and delivery. For example,
Classroom students have the advantage of physical proximity as an
aid to discussion and groupwork but establishing mixed groups of
online learners and classroom-based students might enable the
online learners to search for information online on behalf of the
group, record and display information via online chat or desktop
sharing and contribute in a different role to the overall activity.
Tutors should consider the opportunities presented by the VLE, webconferencing and the internet in accessing and designing learning
resources which stimulate/motivate the learner to acquire new
knowledge and skills.
Linking to student workplaces online present many opportunities to
engage with experienced practitioner or professional workplace
colleagues who may be willing to contribute to discussions,
demonstrate skills, make short presentations to the whole group
online.
SCL course planning,

management and
delivery
Essential
Not
applicable
It is unlikely that a non-SCL course would lend itself to being

delivered in its original format with online students simply linking to
a classroom group. The tutor should review the structure and timing
of planned sessions carefully and ensure that both groups receive
roughly equivalent levels of support and attention.
Online students need a different form of course induction which
needs to be delivered before they actually join an SCL session for
the first time. Online learners will need to be provided with logon
and password information and become familiar with using the webconferencing system before their first proper session begins.
64
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
Depending on subject discipline and students, courses could be

designed whereby students may be required to attend SCL mixed
group sessions, online only sessions and entirely face to face
sessions. Students who miss scheduled campus-based timetabled
course periods which are repeated for other classes could join a
class online to catch up.
SCL course
planning
SCL course
development &
delivery plan
Essential
Not
applicable
When planning a course for SCL delivery, the tutor must consider
resources available for supporting the development/revision of
materials, resources and delivery methods accessible within or
outside the institution.
The participation of online learners in a course previously delivered
entirely on a more traditional face to face basis opens up
opportunities for using the institutional VLE to support
learning/assessment activities.
Tutors should examine the potential for developing materials and
resources in different media and designing delivery patterns which
enable learners to use these independently, in groups and as an
entire class.
Tutors might ask themselves:
65
In what different ways can students be encouraged to

become active learners?
What different media can be applied to the redevelopment of

course resources which would not only satisfy online learner
needs but also significantly enhance the learning experience
of the classroom cohort.
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
Remote
learner
induction
process
Standard webconferencing learner

guide
Essential
Essential
The educational institution should produce a standard set of webconferencing learner guidelines which cover all common aspects of
the system itself, behaviour or netiquette while participating in
sessions and troubleshooting connectivity or other resolvable
system performance issues should these be encountered while
online.
An electronic version of this guide should be freely and readily
accessible online to all students and staff.
Remote
learner
support
materials
Web-conferencing
troubleshooting guide
Essential
Essential
A user troubleshooting guide should cover all known potential

technical difficulties relating to user PC setup, connectivity, PC
settings, webcam & audio hardware/settings.
VLE course materials
Desirable
Desirable
Course-related resources made available on the VLE may enable the

student to access these independently in the event of connectivity
problems during an SCL session. The materials on the VLE may also
be accessed by the classroom cohort between sessions.
E-assessment
facilities
Desirable
Dependant
on
assessment
requirements
Planning for assessment will depend on the assessment

requirements of the course syllabus and delivery plan.
Dependant
on
assessment
requirements
Where assessment requires to be formally invigilated, tutors may

consider using the webcam/audio connectivity of web-conferencing
systems to invigilate an e-assessment remotely, observe students
performing practical assessments or carry out one to one oral
assessments.
E-assessment
E-assessment
invigilation
Desirable
Online quizzes and multiple choice questions, for example, offer

simple and well-tried forms of e-assessment for incorporation into
an SCL programme and are well-suited to formative and summative
assessment for both remote and classroom-based students.
Invigilation of employees may require the recruitment of a local

66
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
supervisor/manager to act as an additional local invigilator for
course assessments.
Accommodati
on
Internet access
Essential
Essential
The room used to deliver to the classroom cohort in an SCL course

requires a fast broadband internet connection.
The locations used by remote online students require internet
access, but bandwidth available is not within the educational
institutions control. Where bandwidth is low, tutors should advise
online students to communicate by audio only.
The course tutor should advertise a minimum bandwidth and
computer setup specification for internet access required to
participate online in course activities.
Electronic whiteboard
/ digital projector
Essential
Not required
The on-campus accommodation used for SCL delivery must provide

either digital projection or electronic whiteboard facilities in order to:
Speakers
Desirable
Not required
Show the classroom cohort electronic presentations and

multimedia resources; and
Display the web-conferencing desktop with direct audiovisual feeds and online chat from remote learners.
Where speakers can be activated in the classroom without causing

audio feedback to online participants, this enables remote learners
to communicate directly with classroom participants without the
need for the tutor to act as intermediary.
Use of speakers rather than headphones by remote participants is
almost guaranteed to result in audio feedback. Participants who
cannot (or will not) de-activate their speakers should be muted.
Open microphone
Desirable
Not required
Ideally, the on-campus location for an SCL session should have an

67
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
open microphone set up which enables remote participants to hear
all discussion and questions/answers within the classroom.
The use of an open microphone would also remove from tutors the
need to wear a headset, providing greater freedom of movement.
In the same way that the settings for open speakers used in the
room need to be adjusted to minimise loss of audio quality resulting
from feedback, so too do open microphone settings need to be
adjusted for optimum audio quality.
Tutors should check the microphone settings applied in the delivery
room, where available, as these may not have been set for an SCL
session.
Evaluation
strategy and
process
Evaluation plan and

supporting evaluation
resources
Essential
Essential
Tutors should establish a strategy and process for comprehensively

evaluating their SCL programmes, taking into consideration
feedback from all participants and support staff involved.
This should include classroom-based and remote participants, any
expert and non-expert support staff involved, employers, peers and
the tutors themselves.
Online questionnaires can be readily set up within the VLE for
students to complete. Structured student focus groups and course
team self-evaluation events should be organised for qualitative
evaluation feedback.
Almost irrespective of the level of planning and preparation which
goes into an SCL programme, the first delivery session by first-time
SCL tutors will almost certainly throw up unexpected and unplanned
difficulties.
Tutors should therefore consider organising a pilot programme which
aims to gather comprehensive evaluative feedback from
68
Requirement
Key elements
SCL
course
delivery
Off-campus
participatio
n
Key considerations
participants in return for a participant sweetener (eg fee reduction
or waiver, lunch). This feedback can then be used to make changes
at an early stage. Where learners know that they are involved in a
pilot and expect the possibility of glitches and interruptions they are
likely to show tolerance and acceptance towards the tutor and adopt
a more positive tone in their critical feedback.
By the second or third course delivery, SCL tutors will begin to feel
more comfortable and confident in using the technology and adapt
to the different presentational and communication styles required.
Donald Steele
SWEET Project Manager
10 January 2012
69
Project Identifier: SWEET

Version: 1
Contact: Donald Steele, Project Manager d.steele@napier.ac.uk
Date: 16 January 2012
APPENDIX H: Synchronous
Collaborative Learning / Big Blue
Button training specification:
Course tutor competences
The SCL/BBB course tutor should be able to demonstrate competency in the full range of knowledge
and skills set out in this checklist. A comprehensive training and skills development programme will
be available to all course tutors from February 2012.
Course planning and design
In planning and designing a course delivered entirely or in part through an SCL/BBB methodology the
course tutor should be able to:
Establish a clear and systematic process for enrolling and inducting remote learners;
Provide ready and comprehensive access to course information and resources, (eg.
course hand book, student BBB guide and helpline contact, learner contact number,
logons/passwords for access to facilities).
Develop a methodology for identifying and satisfying additional learning support
needs within a student cohort.
Establish a clear and systematic process for checking the suitability and functionality
of equipment in off-campus student locations.
Prepare and create appropriate online course support materials which enable
independent learning on a virtual learning environment.
Establish the optimal delivery time for each SCL/BBB session appropriate to the
learning needs of both face to face and remote learners.
Schedule the topics to be covered in each session, with contingency plans for loss of
BBB functions or online student connection
Develop and apply appropriate strategies for enabling interaction between face to
face and on line learners within an SCL/BBB session..
Develop and apply appropriate strategies for managing the interaction between face
to face, on line learners and tutors within an SCL/BBB session.
Identify, develop and deploy interactive learning resources which support the
objectives of each SCL/BBB session.
Prepare formative and summative assessments which learners can complete using
SCL/BBB while meeting the course requirements.
Establish an effective evaluation methodology for each SCL/BBB session and the
overall course.
Set clear objectives for the learning experience and evaluate their achievement;
Review and redevelop training materials to ensure a sound pedagogical delivery to
both online and classroom based learners;
Course Delivery
In delivering a course delivered entirely or in part through an SCL/BBB methodology the course tutor
should be able to demonstrate prior to the first session the competency to:
Student support
Identify and support learners with additional support needs;

Page 70 of 71
Document title: JISC Final Report Template

Last updated : Feb 2011 - v11.0
Project Identifier: SWEET

Version: 1
Contact: Donald Steele, Project Manager d.steele@napier.ac.uk
Date: 16 January 2012
Interact equally with both online and face to face learners within an SCL/BBB.
Consolidate the learning at the beginning and end of each session
Internet usage
Access BBB using the Firefox web browser;

Have undertaken a viewing of the BBB training videos on the BBB training web page
Wear a single headphone and microphone and simultaneously communicate directly
with online learners and classroom-based learners.
Access courses on the Discover VLE
Open and navigate between two or more web-browser tabs when using multimedia
resources;
Create and save PDF versions of course presentations and documents;
Browse to and upload presentations from networked storage to BBB;
Navigate and create links to course-related web pages;
Big Blue Button
Identify and make appropriate use of Big Blue Button icons;

Allocate the presenter role to different participants within a BBB session;
Share the presenter web camera with other participants;
Control the usage of computer speakers among participants, ensuring these are
switched off when interfering with audio reception;
Share and adjust the levels of the presenter microphone;
Isolate, reconnect and adjust the presenter microphone to the levels required for
optimum sound quality, with minimum feedback;
Control the volume of the presenter microphone;
Identify and mute additional microphones built into the presenters hardware in order
to prevent feedback;
Use and demonstrate usage of the raise hand function;
Use, read and respond to online chat effectively within an SCL/BBB session;
Mute online participants where required;
Share the presenter desktop with other course participants;
Arrange webcam feeds from participants as required around the BBB presenter
screen;
Presenting
Act as an effective learning facilitator between online and classroom-based learners;

Effectively and simultaneously interact with on line and face to face learners within a
single SCL/BBB session;
Adopt a presenter standing position which avoids turning away from either online or
classroom-based learners;
Adopt a presenter delivery position which prevents entanglement with headphone,
microphone and computer cables;
Troubleshooting
Recognise and troubleshoot common technical faults/issues associated with the BBB
system;
Troubleshoot common remote learner connectivity and computer performance
problems
Access and liaise with expert technical support staff for assistance with using BBB
Document title: JISC Final Report Template

Last updated : Feb 2011 v11.0
Page 71 of 71

Sweet Final Report FINAL

Uploaded by

Copyright:

Available Formats

You might also like

Sweet Final Report FINAL

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sweet Final Report FINAL

Uploaded by

Copyright:

Available Formats

JISC Final Report

Synchronous Web Enabled Employee Training

Jewel & Esk College

Wendy MacAdie, Head of Centre: Community Engagement

Project Web URL

ELVON (East Lothian Voluntary Organisations Network)

JISC Learning and Teaching Innovation Grants: SWaNI FE (10/10)

This report is for general dissemination

Submission date extended in agreement with JISC Programme

MAIN BODY OF REPORT.............................................................................5

PROJECT OUTPUTS AND OUTCOMES...............................................................................5

IMPLICATIONS FOR THE FUTURE..............................................................21

3 Main Body of Report

Brief Description and URLs (where applicable)

BigBlueButton acceptance test

A user acceptance test template was developed and applied when

Integrated open source

A fully operational Moodle/BigBlueButton system has been in service

Technical report and specification

SCL tutor comptences

A tutor competences specification aimed at BigBlueButton system

A tutor guide to BigBlueButton, covering both technical system usage

A student guide to BigBlueButton, covering both technical system

Test report on REHIS

A detailed evaluation of the course delivery model and experience has

Package of course resources

A revised presentation, redesigned for the SCL delivery methodology

Employer requirements checklist

A guide and checklist of employer requirements for supporting

A case study (Synchronous Collaborative Learning case study 1:

Output / Outcome Type

Brief Description and URLs (where applicable)

REHIS pilot SCL cohorts trained

18 classroom-based and 8 online students completed the REHIS

Student evaluation process

A student evaluation process was developed which included an

The SWEET Project methodology provides a template for other

Learning, support and

A Synchronous Collaborative Learning: course planning, setup and

Remote candidate e-assessment

A detailed methodology for managing formal assessment at remote

3.2 How did you go about achieving your outputs / outcomes?

the costs of releasing staff;

Project management structure, comprising:

Review the project risk assessment;

Review and approve deployment and expenditure of the project budget;

Agree actions required to address specific issues arising.

Project implementation phases

Install, configure & test

Virtual Learning Support & Development

Rack & stack server; run hardware tests

ICT Systems Manager (Mar)

Install & test operating system & BBB

ICT Systems Manager (Mar)

Install, configure & test BBB/Moodle

Virtual Learning Support & Development

Research, install & tests BBB server

ICT Systems Manager (Mar)