An Evaluation

Report for
Quality and
Participation
within Nursing
Fundamentals
200.
By Aaron F Nelson

NIGHTINGALE
COLLEGE
INTERACTIVE
MODULE
EVALUATION
REPORT

This is an official evaluation submitted by Aaron Nelson for

the stakeholders of Nightingale College on May 9, 2014.

Contents
YSumm
Description of the program evaluated........................................................................3
Program Objectives................................................................................................. 4
Program Components.............................................................................................. 4
Evaluation Method........................................................................................................... 4
Participants............................................................................................................. 4
Nightingale Evaluation Team...................................................................................5
Procedures............................................................................................................... 5
Data Sources........................................................................................................... 6
Results........................................................................................................................ 7
Discussion.................................................................................................................. 9
Project Cost.............................................................................................................. 10
Appendices............................................................................................................... 10

Summary
Nightingale College is transferring into a competency based curriculum. As part of this
transformation the Curriculum management team have implemented interactive modules to
replace its old way of delivering lectures through taped video casting through BlackboardCollaborate. The school started the interactive program a year ago. After much research, the
team decided on Storyline by Articulate. The developers picked Storyline because of the ease of
building a course, the quality of the interactive modules, and the ease of implementation into the
LMS. Each module consists of a weekly unit that includes objectives and outcomes for each
specified chapters of the nursing curriculum. The objectives are aligned with the NCLEX board
exams, which nurses must pass to become registered nurses.
The interactive modules have been implemented into Nursing Fundamentals for a year. They
were created by Karen Burton, the Colleges curriculum manager and the e-development team
which is headed by Blake Halladay, the Colleges e-Developer. The modules include crossword
puzzles, games, quizzes, character animations, videos, and other media that provided a unique
way of introducing basic nursing fundamentals.
The purpose of the evaluation report is to give the curriculum management team and
stakeholders of the College information on how well we are doing in the quality of our product,
and participation among the students in the interactive modules. The major steps of the
evaluation included meetings with course developers, students, and administrators. Evaluation
was gleaned from data that was provided from surveys and information reports.
The evaluation found that there are many factors that need to be improved in order for the
objectives to be met. The College will take the information it has received from the evaluation
report and prepare a plan of action. The results of the evaluation will be detailed at the
conclusion of the evaluation report.

Description of the program evaluated

The intended users of the interactive modules include every course and student at Nightingale
College. Currently, the student body is around 100 students. The College has twenty-seven
students enrolled in the Fundamentals Nursing course, where we will conduct the evaluation.
On average, students participate in 15-18 interactive modules per course. They will be graded on
participation and evaluation, which will be included at the end of every interactive module.
Those students who excel at the interactive modules (an overall grade of 80%) will be given the
option of skipping the final examination.
The reason why the College decided to pursue interactive modules in place of webcast lectures is
because the College wanted more participation and involvement with course content. The
instructors are given data concerning how much time the students spend in the course, how long

it takes for the student to do the interactive module, and test scores for the unit reviews of each
interactive module. These reports are available for instructors through Moodle.

Program Objectives
There are two program objectives:

The first objective is to evaluate the quality of fifteen interactive modules in Fundamental
Nursing. The design team is interested to see if the modules are easy to navigate, have
good designs, graphics, and videos, and meet the objectives outlined for each unit
chapter.
The second objective is to evaluate participation in the modules. Since participation in
the interactive modules counts toward a large portion of the students grade, the College
wants to know the amount of time students are participating in the interactive modules,
and how they are influencing overall grades.

Program Components
The intended purpose of Storyline by Articulate is to provide students with polished modules that
students are able to use. The modules may include quizzes, media videos, screen recordings, and
drag-and-drop interaction. The developers are also able to easily build complex interaction
characters using a host of templates, characters, and animations. For the Nursing Fundamentals
course, fifteen modules have been built. Before the program, instructions will be given to the
beginning nursing students on how to navigate the modules.
Evaluation Method
Our goal is to have an independent and neutral evaluation process. We will be using the goalbased model for evaluation. The evaluation method includes three partsplanning, evaluation,
and presentation.
Here is a description of the evaluation method:
1. Aaron Nelson, the head of the evaluation team met with the Director of Curricula to
decide on participants of the team. Objectives were decided on, surveys were written,
and dates were planned.
2. The information will be analyzed, surveys will be handed out, and the evaluation report
will be written.
3. We will deliver our findings to Mikhail Shneyder, the CEO of the College.

Participants
Student body: The Fundamentals Nursing 200 course is made up of first year students in the
AND (Associate Degree Nursing) program. Before entering into Nightingale College, the
students need to complete fifteen general education credits in order to participate in the program.
The current course includes twenty-four participants. Most of these students have previous

experience in the nursing field; however, the College does allow students who are inexperienced.
Its also important to point out that many of students have little to no experience using
technology. Extensive training and support is provided for help with technology issues.

Nightingale Evaluation Team

The following people will be the main evaluators and respondents for the official final evaluation
report:
Aaron Nelson (Director of Library Services): Aaron Nelson has over three years of library
experience. He received his BA from Weber State University and an MLS from Emporia State
University. Mr. Nelson has been a part of Nightingale University for three years. Currently, Mr.
Nelson is pursuing MET (Master in Educational Technology from Boise State University. He
has over six months experience in evaluation experience.
Karen Burton (The Director, Nursing Curricula and Academics):, Dr. Karen Burton, PhD,
RN, CNE is responsible for the design, development, and consistent implementation of nursing
curriculum and academic programs across the current and future programs at the College. She is
currently collaborating with the faculty to transition the ADN Program curriculum from
traditional to concept-based. Dr. Burton reports to the President and CEO, and manages all
library and instructional design efforts at the College. She currently holds her Certified Nurse
Educator (CNE) certification and received both her masters and doctoral degrees specifically in
nursing education. She has extensive experience with curriculum design and teaching
methodologies. Dr. Burton holds a doctorate in philosophy (PhD) with a major in nursing
education from the University of Northern Colorado; Greeley, Colorado.
Sue Jero (ADN Program Manager): Ms. Jero has full responsibility for the day-to-day
operations of the ADN Program. Further, Ms. Jero is a member of American Organization of
Nurse Executives (AONE), National League of Nursing (NLN), the Academic Leadership
Committee (UONL), and is a member of Ogden Chamber of Commerce. Ms. Jero has been with
the College since inception.
Blake Halladay: Mr. Halladay holds an associate degree with a major in nursing and has an
extensive background of professional experience in new partner acquisition and key partner
relationship management roles. Mr. Halladay comes to the College from one of the largest and
most respected corporate education programs in the country, the Learning and Development
group at Fidelity Investments, a trillion dollar financial corporation. During his tenure at the
Learning and Development group, Mr. Halladay operated in various capacities that included
small group education in a physical classroom, presentations to large groups of employees,
project management, relationship management, instructional design, and course development.

Procedures
As this was the first time implementing Storyline by Articulate, there were many obstacles that
needed to be overcome. The e-Developer (Blake Halladay) had to be trained on the tool. The

students had to be trained on how to navigate through the modules. The faculty had to be trained
on how to design the content that the e-Developer put into the interactive module. Naturally,
there was a learning process that was new to the College.
The evaluation portion of Storyline began in February and concluded in May. This gave the
Nightingale Evaluation team plenty of time to have enough modules developed. It also gave the
students enough time to become familiar with the content. Below is a timeline of the evaluation
procedures that the team followed in order to develop the evaluation report.

Procedure

Date

Planning: Aaron Nelson, the head of the Evaluation Team met with
Karen Burton, the director over the Curriculum/Design Team.
During this meeting they will decided what other members to
include on the Evaluation Team. They will also discuss roles, data,
objectives, evaluation report, and other pertinent data required for
the report.
The Evaluation Team met to discuss objectives for the interactive
modules. The Evaluation Team decided on two objectives that deal
with quality and participation. These two objectives were decided
upon after much discussion.
Evaluation: The Evaluation Team and e-Development team met
with class ambassadors to discuss the new interactive modules. The
ambassadors went through the first 5 unit modules and discussed the
quality, videos, interface, games, and other factors that they liked or
disliked about the interactive modules.
The Evaluation Team meets to create survey questions. Its decided
that the survey will be a short five question survey that will be
designed to answer the Evaluation Teams objectives. Its also
decided that the survey will be handed out with the other course
surveys during the last two weeks of the course semester starting
April 14.
The Course surveys are created in www.surveymonkey.com and
links are sent to the students in the Nursing 200 course.
The evaluation report is written by Aaron Nelson. Once the report is
examined for his EDTECH 505 course at Boise State University, it
will be rewritten with any advice from the instructor.
Presentation: The final evaluation will be delivered to Mikhail
Shneyder, Campus CEO, along with any members of the design
team.

February 3, 2014

February 24, 2014

March 10, 2014

March 27, 2014

April 14, 2014

April 21-May 9th, 2014

June 16th, 2014

Data Sources
The College employed a few data tools to gather information on the factors that are stated in its
objectives. Below is a table that will show each factor, including the tool that we used to
measure each factor. An appendix of all of the information gathered can be found at the
conclusion of the report.
Factor
Performance (Test Reviews)

Participation

Quality

Information Tool
Storyline, SCORM, Moodle. The evaluation team used these
three tools to measure performance. At the end of every unit test
scores are evaluated using a template from Storyline. Using a
technology tool called SCORM; the scores are then transferred
over to Moodle, where the team is able to evaluate test reviews
and scores in Excel spreadsheets.
Moodle. The LMS tracks how often students use the interactive
module. If a sign of Not Attempted or Incomplete shows up,
it clearly shows that the student didnt participate, or partially
participated in the unit. Also, it shows when the student started
and ended each unit. If the student started the module a day
before the module was due, it clearly shows that the student
wasnt actively engaged in the interactive module.
Survey Monkey. For the quality factor, the students are evaluated
using surveys created on www.surveymonkey.com. The surveys
query the students on graphics, videos, interactive games, etc.
The evaluation team decided on using questions in the Likert
Scale form.

Results
The survey gave results for both objectives. The survey was designed according to the Likert Scale. All
27 students participated in the survey. Here is a summary of these findings:
Survey Question

1. Overall, I enjoyed the interactive

modules for my Nursing 200
course.
2. It was easy for me to log into
Moodle so that I could do the
interactive module.
3. The graphics, pictures, and media
content helped me understand the
content for Nursing 200
Fundamentals.
4. The games (crossword puzzles,

Findings
The survey scored a 7.67. Overall, students enjoyed the
modules compared to lectures.
The survey scored 7.56. There were very few problems
that were pointed out by students. Most of them had an
easy time logging into Moodle.
The survey scored 7.67. Overall most students liked the
graphics and quality of the interactive modules.

The survey scored 7.73. Most students liked the

matching, etc) were very helpful and

helped me retain the nursing content:
5. I prefer the traditional way of
lecturing (BlackBoard Collaborate)
over the new interactive modules.

interactive content created by the development team.

The survey scored 3.33. Most students do not prefer
traditional online lectures to interactive modules.

The Moodle Reports measured participation and gave results for our second objective:

Unit

Dates
Taken

Incomplete/Did
Not Take

Take
n

Unit 1

01/0701/12
01/1301/19
01/2001/26
01/2702/02
02/1002/19
02/2002/26
02/2403/03
03/0403/11
03/1003/16
03/1003/23
03/1703/23
03/2403/30
03/3104/06
04/0704/13
04/1404/20

Unit 2
Unit 3
Unit 4
Unit
6.3
Unit 7
Unit 8
Unit
8.1
Unit 9
Unit
10
Unit
11
Unit
12
Unit
13
Unit
14
Unit
15

19

Received a
Failing Score on
Module
17 Students

Received a
Passing Score
on Module
10 Students

15

12

14 Students

13 Students

13 Students

14 Students

27

12 Students

15 Students

22

10 Students

17 Students

23

16 Students

11 Students

19

12 Students

15 Students

15

12

23 Students

4 Students

23

7 Students

20 Students

22

8 Students

19 Students

21

9 Students

18 Students

19

15 Students

12 Students

12

20 Students

7 Students

16

16 Students

11 Students

Discussion
The purpose of the reports and survey were to gather information on how well the College is
meeting its objectives for quality and participation. According to the survey, the students are
satisfied with navigation, the graphics, videos, and accessibility. They are finding little to no
difficulties logging into the LMS, viewing the interactive modules, and doing the interactive
module.
The Moodle Reports give valuable information on participation. They show when students
access the interactive modules, grades, and participation. They were easy to access and analyze.
The sources reveal that there were also many problems. It shows that many students didnt
realize the importance the interactive modules were to their overall grades. As this was a pilot
program, this was taken into account for final grades. Its also important to note that the reports
were confusing to read since the e-developer had added many inactive students and faculty
members to the modules.
The results for the Moodle Reports clearly give an indicator of low participation. There were
many students who consistently had incomplete or did not take the modules. There were also
many students who consistently failed the modules. On the other hand, there were also other
students who consistently took the modules and received passing grades. Since the modules are
based on readings within the textbooks, there is a clear indication that some students never read
the textbooks. Not surprisingly, those who fully participated in the interactive modules, also
show a clear indication of passing our end of level HESI exams (our Nursing exams). The reports
also indicate that there were problems rolling out the program. For instance, Units 8.1 and 9,
show that there was close to no participation in the interactive modules.
The College must make the importance of doing the interactive modules clear to the students
beginning with the first day of class. After speaking with the instructors and designers, it was
revealed that grades and indicators for doing the interactive modules werent shown to the
students on Canvas (the other LMS system that students use to look at grades.) The reason they
put the modules on Moodle is because Canvas isnt SCORM compatible. Its highly suggested
to the College that they do away with Canvas. Having two different LMSs will greatly confuse
students and instructors.
The data also suggest that there needs more time devoted to preparing the interactive modules
into Moodle. The weeks where there were no modules prepared had an effect on the overall
participation. A suggestion would be to have a meeting to reevaluate how the developers
implement and create the interactive modules. Students also need to be taught how the
interactive modules work and who they need to contact if there is a problem with the modules.
Overall, the data showed that the quality of the modules is good. The best indicator for this will
be in the final Fundamentals HESI examinations for the College. (Test score data will not be

available until the middle of May.) If HESI examination scores are low, the College must reevaluate interactive modules in place of traditional lectures. Based on student surveys, they
highly prefer the modules over the traditional lectures on BlackBoard Collaborate. With a few
corrections in the way that the College implements the program at the beginning of the semester,
the hope is that participation levels will increase.

Project Cost
The cost for the evaluation report is based on a daily rate of $500.00 per a daily rate. Below
shows the dates that Aaron Nelson worked on the evaluation report:
February 3, 2014
February 24, 2014
March 10, 2014
March 27, 2014
April 14, 2014
April 21-May 9th, 2014
June 16th, 2014
Full Cost = $10,500 for services rendered. (21 Days X $500= $10,500)
Expenses=$4,000
Total Cost = $14,500
The expenses include travel, copies of report, consultations, etc. For a full version of expense
report, please view receipts, reports, etc. in Appendices.

Appendices
Appendix 1=Survey Summary for the 27 Students.
Appendix 2-14=Moodle Reports.
Appendix 3=Expense Reports.