International Journal of Nursing Studies 51 (2014) 136–149

Contents lists available at SciVerse ScienceDirect

International Journal of Nursing Studies

journal homepage: www.elsevier.com/ijns

Review

Impact of e-learning on nurses’ and student nurses knowledge, skills, and

satisfaction: A systematic review and meta-analysis
Mari Lahti a,*, Heli Hätönen a, Maritta Välimäki b
a
University of Turku, Faculty of Medicine, Department of Nursing Science, Turku, Finland
b
Southwest Hospital District, Turku, Finland

A R T I C L E I N F O A B S T R A C T

Article history: Objective: To review the impact of e-learning on nurses’ and nursing student’s knowledge,
Received 5 February 2012 skills and satisfaction related to e-learning.
Received in revised form 18 November 2012 Design: We conducted a systematic review and meta-analysis of randomized controlled
Accepted 19 December 2012 trials (RCT) to assess the impact of e-learning on nurses’ and nursing student’s knowledge,
skills and satisfaction. Electronic databases including MEDLINE (1948–2010), CINAHL
Keywords: (1981–2010), Psychinfo (1967–2010) and Eric (1966–2010) were searched in May 2010 and
e-Learning again in December 2010. All RCT studies evaluating the effectiveness of e-learning and
Nurse
differentiating between traditional learning methods among nurses were included.
RCT
Data extraction and quality assessment: Data was extracted related to the purpose of the trial,
Traditional learning
sample, measurements used, index test results and reference standard. An extraction tool
developed for Cochrane reviews was used. Methodological quality of eligible trials was
assessed.
Data synthesis: 11 trials were eligible for inclusion in the analysis.
Results: We identified 11 randomized controlled trials including a total of 2491 nurses and
student nurses’. First, the random effect size for four studies showed some improvement
associated with e-learning compared to traditional techniques on knowledge. However,
the difference was not statistically significant (p = 0.39, MD 0.44, 95% CI 0.57 to 1.46).
Second, one study reported a slight impact on e-learning on skills, but the difference was
not statistically significant, either (p = 0.13, MD 0.03, 95% CI 0.09 to 0.69). And third, no
results on nurses or student nurses’ satisfaction could be reported as the statistical data
from three possible studies were not available.
Conclusion: Overall, there was no statistical difference between groups in e-learning and
traditional learning relating to nurses’ or student nurses’ knowledge, skills and
satisfaction. E-learning can, however, offer an alternative method of education. In future,
more studies following the CONSORT and QUOROM statements are needed to evaluate the
effects of these interventions.
ß 2013 Elsevier Ltd. All rights reserved.

What is already known about the topic?

 The use of e-learning and computer assisted methods is

* Corresponding author at: University of Turku, Department of Nursing rapidly increasing in medical and nursing education.
Science, Vartiovuorenkatu 3Ba22, 20700 Turku, Finland.  The use of innovative learning methods has been shown
Tel.: +358 41 458 14 55.
to be promising in medicine to improve knowledge
E-mail addresses: melaht@utu.fi, marielinalahti@gmail.com
(M. Lahti).
among learners.

0020-7489/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.ijnurstu.2012.12.017
 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
 There is still a lack of evidence about the impact of
computer assisted and e-learning methods in nursing
education.
What this paper adds
 There is a lack of systematically conducted RCT studies
comparing the effects of e-learning and traditional
learning methods in nursing education.
 In nursing education, e-learning is not superior to
traditional learning method but can be used as an
alternative method in nursing education.
 e-Learning should be used with a good understanding of
learners’ needs.
1. Introduction
Lifelong learning is one of the priorities in the European
educational area (European Commission, 2010). Informa-
tion and communication technology (ICT) opens doors to
new innovative methods to deliver education across the
lifespan (European Commission, 2011). The use of e-
learning is a rapidly growing form of education and a new
way of delivering education in general (Digital Agenda
Assembly, 2011; Commission of the European Commu-
nities, 2000, 2001, 2003). This is also the case in health
care, where the Internet and Worldwide Web (WWW)
have expanded the opportunities for flexible, convenient,
and interactive education (Cook et al., 2010a, 2008; Belcher
and Vonderhaar, 2005; Wutoh et al., 2004) for educational
provision and health care professionals (Lowry and
Johnson, 1999).
e-Learning has been described as a dynamic, innovative
and rich way to provide learning opportunities (Belcher
and Vonderhaar, 2005). Students can access a class through
a website and participate in lectures and group discussion
in real time. Materials may also be provided asynchro-
nously; students access the website, follow lectures or
complete assignments according to their own schedules
(Simpson, 2003.) In the recent literature there are a few
systematic reviews of research on e-learning (Cook et al.,
2008, 2010a,b; Curran and Fleet, 2005; Cobb, 2004; Wutoh
et al., 2004; Schittek et al., 2001). In general, the benefits
reported for e-learning are flexibility, accessibility, satis-
faction and cost-effectiveness (Smith, 2005; Wutoh et al.,
2004; Ward et al., 2008). A meta-analysis by Cook et al.
(2010a) showed that e-learning can increase students’ own
control over the content, place and time of learning.
Furthermore, it can help students gain knowledge and
skills faster than traditional instructor-led methods (Cook
et al., 2008).
The review by Cook et al. (2008) and Wutoh et al. (2004)
focused on the effectiveness of e-learning. Cook et al.
(2008) reported positive results in their meta-analysis
comparing e-learning intervention to non-intervention.
Wutoh et al. (2004) also report in their review that e-
learning for delivering continuing medical education is as
good as other methods in transferring knowledge. In
addition, Cook et al. (2010b) compared e-learning inter-
ventions to other types of computer based educational
137
interventions. They showed that interactivity, practice
exercises, repetition, and feedback improved learning
outcomes while using e-learning interventions (Cook
et al., 2010b).
However, reviews of the effectiveness of e-learning
have focused mostly on professionals working in the field
of medicine or health care in general (Cook et al., 2008,
2010a; Curran and Fleet, 2005; Cobb, 2004; Wutoh et al.,
2004). To the best of our knowledge only one review has
been conducted on e-learning in general (Schittek et al.,
2001). No systematic reviews of e-learning and its
effectiveness have been done among nurses or student
nurses by meta-analytic methods although nurses are one
of the largest groups of professionals in health care
(Eurostat, 2011). Therefore it is important to focus on
the impacts of e-learning on this professional group. In
addition, as in the concern expressed by Curran and Fleet
(2005), Curran et al. (2010), too often evaluative education
studies rely on participants’ satisfaction level while more
energy should be put into knowledge transfer, skills
improvement and the changes in practice after educational
interventions. The importance of focusing on a variety of
outcomes when evaluating educational training such as e-
learning is also noted in Kirkpatrick’s model of learning
outcome evaluation (Galloway, 2005). Kirkpatrick’s model
has four levels to evaluate educational training: (1)
reaction, i.e. student satisfaction, (2) learning, i.e. knowl-
edge increase, (3) behavior, i.e. change in practice and (4)
results, i.e. final outcomes (Kirkpatrick, 1998; Galloway,
2005). Therefore the focus in this review was on the impact
of e-learning on knowledge, skills and satisfaction among
nurses and nursing students.
2. Objective
This systematic review aims to investigate the impact of
e-learning on knowledge, skills and satisfaction among
nurses and nursing students compared to traditional
education methods.
3. Methods
The research consists of a systematic review (Higgins
and Green, 2011).
3.1. Data sources and searches
We electronically searched MEDLINE (1948–2010),
CINAHL (1981–2010), PsychInfo (1967–2010), Eric
(1966–2010) for publications in English in May 2010.
The search was updated in December 2010. No restrictions
were placed on date of publications and each database was
searched as far back as possible (Savoie et al., 2003). This is
recommended method when search terms has varied in
past history or showed a huge diversity which is the
situation with the concept of the e-learning (Egger et al.,
1997).
As each database has its own unique indexing terms,
individual search strategies were developed for each
database. Consideration was given to the diverse termi-
nology used for e-learning as this would influence the
138 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
identification of relevant trials. Focus was in finding a
relevant MeSH terms or Subject Headings in different
databases related to this topic as e-learning is not a listed for
Subject Heading. Medical informatics was also consulted to
ensure that the right search terms were used. The search
strategy used in English was as follows: ‘‘(Computer-
Assisted Instruction/or computer assisted.mp. or online.mp.
or www.mp. or internet.mp. or Internet/or computer
based.mp.) and (learning/or problem-based learning/or
problem solving/or learn*.ti,ab. or cal.ti,ab. or Knowledge/
or knowledge.ti,ab.) and (psychiatric nurs*.mp. or Psychia-
tric Nursing/or Nurses/or nurse*1.ti,ab.) and (Random
Allocation/or random*.mp. or randomized controlled
trial.pt.) limit to (abstracts and English language) (Savoie
et al., 2003; University of York, 2008). ‘‘Psychiatric’’ was
included as a search term to ensure that all relevant
literature will be achieved. In addition, the reference lists
and bibliographies of retrieved articles were reviewed to
identify any additional items. To complement the search
strategies keyword searching of the World Wide Web and
Cochrane Library was also conducted (Khan et al., 2003).
3.2. Inclusion criteria
We used pre-defined criteria to include studies in our
review and studies fulfilling these criteria were included.
We included studies with nurses or nursing students used
as a study population. The intervention used was e-
learning. It was defined as computer assisted learning, use
of CD-ROM and online learning as these would refer to be
nuanced variations for e-learning, while traditional
education included contact teaching, classroom lecturing,
and text-based learning. Outcomes listed were increase of
knowledge and skills and satisfaction (Khan et al., 2003;
University of York, 2008).
To be included, the studies had to be randomized
controlled trials (RCT), controlled clinical trials (CCT),
controlled before and after (CBA) studies using e-learning
among nurses or nursing students. We excluded studies
using simulation, mobile phones and mannequins, like-
wise studies not published in English (Khan et al., 2003).
3.3. Data extraction
The extraction tool developed for Cochrane reviews was
used. A reviewer (ML) independently extracted data
related to the following issues: purpose of the trial,
sample, details of the intervention, outcomes and mea-
surements used, follow-up time, index test results and
reference standard. If any data were missing from the
report of the trial, attempts were made to obtain these by
contacting the authors. Each citation was then assessed
against the inclusion/exclusion criteria independently by
one reviewer (ML). The full texts of studies eligible for the
review were then obtained. For studies with unclear titles
and abstracts, the full text was also obtained. Decisions to
include a publication in the review were made by the
reviewers (ML, MV, HH). This was followed by evaluation
of the full text of all papers retrieved. In case of any
disagreement the paper was discussed with other member
of research group (Khan et al., 2003).
3.4. Assessment of the methodological quality of the included
trials
The methodological quality of the included studies was
assessed using process and criteria based on those of the
Cochrane Collaboration Handbook for Systematic Reviews
of Interventions (Higgins and Green, 2011). This assesses the
methodological quality in the following six domains: (1)
Sequence generation – was the allocation sequence
adequately generated? (2) Allocation concealment – was
allocation adequately concealed? (3) Blinding of partici-
pants – was adequate knowledge of the allocated interven-
tion adequately concealed from the outcome assessors? (4)
Incomplete outcome data – was this adequately addressed
for each outcome (this includes differential attrition
between groups)? (5) Selective outcome reporting – are
the reports of the study free from suggestions of selective
outcome reporting? 6) Other sources of bias – was the study
apparently free from other problems that could subject it to
put it at a high risk of bias? The methodological quality of
these domains was assessed by using following scoring: (1)
‘low risk of bias’, when plausible bias unlikely alter the
results, (2) ‘unclear risk of bias’ when plausible bias raises
some doubt about the results and (3) ‘high risk of bias’ when
plausible bias seriously weakens confidence in the results
(Higgins and Green, 2011).
To deal with publication bias the Cochrane Handbook
for Systematic Reviews of Intervention (Higgins and Green,
2011) was used to guide the process. Attempts to locate the
protocols of the studies included were made, but none of
the studies included had a protocol. As no protocols were
used, lists of outcomes mentioned in the method section
were used in the study and compared to the results
actually reported. The methodological quality of eligible
trials was assessed independently by three reviewers (ML,
HH, MV).
3.5. Quantitative data synthesis
The data from the studies included were entered into
Review Manager 5.1 (RevMan), which is the software used
for preparing and maintaining Cochrane Reviews. The data
analysis was divided into two phases. First, the analysis
explored the descriptive characteristics of the individual
studies included. Second, for continuous outcomes we
estimated mean difference (MD) between groups (Moore
and McCabe, 2003). When scales of very considerable
similarity such as knowledge tests were used, we
presumed there was a small difference in measurement
and combined the measures. This decision was made to
answer the overall question of whether there is evidence
that e-learning can work (Higgins and Green, 2011). In this
approach standard deviations were used together with the
sample sizes to compute the weight given to each study. In
general, studies with small standard deviations are given
relatively higher weight while studies with larger standard
deviations are given relatively smaller weights. This is
appropriate if the variation in standard deviations between
studies reflects differences in the reliability of outcome
measurements (Higgins and Green, 2011). Then we
calculated effect sizes from mean differences of post test
 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149 139

scores (Ray and Shadish, 1996). Random effect was used 4. Results
instead of fixed effect because random effect allows the
outcomes of studies to vary more than fixed effects. In 4.1. Trial flow
other words, random effects can be seen to be a more
A meta-analysis profile summarizing the flow diagram
natural way of explaining outcome (Ades and Higgins,
is presented in Fig. 1.
2005). Heterogeneity was checked by calculating using I2 –
A total of 158 publications were identified from the
square statistics. This provides an estimate of the
databases. A review of the reference lists and bibliogra-
percentage of variability due to heterogeneity rather than
phies of the items retrieved identified 19 additional clinical
change alone. Where the I2 – square estimated was greater
trials relevant to the topic. Out of the 177 publications, 26
than or equal to 50%, it was interpreted as indicating the
were duplicates. All together from 151 publications, 132
presence of high levels of heterogeneity (Higgins and
publications were excluded because they did not meet the
Green, 2011; Higgins et al., 2003; Higgins and Thompson,
inclusion criteria. Thus, 19 studies were read in full. Eight
2002).
studies were excluded due to wrong population (n = 3) and
No assumptions about loss to follow-up were made for
a lack of randomized design (n = 5). Thus a total of 11
continuous data and the results were analyzed for those
studies were included in the quantitative data analysis
who completed the trial, since the use of methods such as
(Fig. 1).
the last observation carried forward (LOCF) introduce
uncertainty about the reliability of any result (Leucht et al., 4.2. Methodological quality of the included trials
2007). The high number of the loss to follow-up can be a
threat for the validity of the results. However, even The methodological quality of the 11 publications
relatively high rates of loss to follow-up will result bias included in the review varied. Incomplete details in
only if the number lost is imbalanced between groups reporting the sequence generation and allocation con-
(Guyatt et al., 2011). cealment were the main reasons for decreasing the
Idenficaon

Records idenfied through Addional records idenfied

database searching through other sources
(n = 158) (n = 19 )

Records all together

(n = 177)

Records aer duplicates removed

Screening

(n = 151)

Records screened by tle Records excluded by tle

(n = 151) (n = 89)

Records screened by Records excluded by abstract

abstract (n = 43)
Eligibility

(n = 62)

Full-text arcles assessed Full-text arcles excluded,

for eligibility with reasons
(n = 19) (n = 8)
Included

Studies included in
quantave synthesis
(meta-analysis)
(n = 11)

Fig. 1. Flow diagram (Moher et al., 2009).

140 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
Fig. 2. Risk of bias based on Cochrane Systematic Reviews of Interventions
(Higgins and Green, 2011).
methodological quality of included studies. None of the
studies were blinded or no attempt was made at blinding.
This may be due to the nature of the intervention, where
blinding would be difficult to achieve. In addition,
incomplete details in selective outcome reporting low-
ered the quality as none of the studies reported the use of
protocols. However, this was assessed as ‘low risk of bias’
because it was assumed that the published reports
included all outcomes (Higgins and Green, 2011) (Fig. 2).
4.3. Study characteristics
This review includes 11 studies published between
2004 and 2010. All studies were reportedly randomized,
although the description of the allocation varied. All 11
studies used a parallel study design. However, one
employed a cross-over design (Cega et al., 2007)
(Table 1). Power calculations were conducted in two
studies (Cega et al., 2007; Paladino and Peres, 2007), and
one study (Horiuchi et al., 2009) reported some guidance
used to calculate adequate power for the sample size.
However, eight studies did not report any power calcula-
tion to estimate the adequate sample size. Nevertheless,
meta-analysis of individual studies used in this review
increases statistical power by reducing the standard error
of the weighted average effect size (Cohn and Becker,
2003).
Information about participants totaled 2491 nurses and
nursing students. Three of the studies included had
student nurses’ (n = 618) (Beeckman et al., 2008; Bloom-
field et al., 2010; Cega et al., 2007) and eight of them had
nursing staff (n = 1873) (Bloomfield et al., 2010; Harrington
and Walker, 2004, 2006; Hsiao Sheen et al., 2008; Irvine
et al., 2007; Mcvey et al., 2009; Paladino and Peres, 2007;
Tsai et al., 2004). Overall, the study size varied from 42
(Hsiao Sheen et al., 2008) to 1294 (Harrington and Walker,
2004) participants (Table 1).
Three studies used previously tested and validated
outcome measurements (Bloomfield et al., 2010; Cega
et al., 2007; Harrington and Walker, 2004) and one study
used partially validated measurement (Mcvey et al., 2009).
Seven studies used self-developed instrument, i.e. a
questionnaire developed for that particular study. Nine
studies had primary outcome measure as knowledge
(Bloomfield et al., 2010; Cega et al., 2007; Harrington and
Walker, 2004, 2006; Horiuchi et al., 2009, Irvine et al.,
2007, Mcvey et al., 2009; Paladino and Peres, 2007; Tsai
et al., 2004), while two studies had nursing skills as a
primary outcome (Beeckman et al., 2008; Hsiao Sheen
et al., 2008) (Table 1). All outcome measurements used the
same kind of scoring where higher score indicates better
knowledge or skills achieved.
4.4. Description of the interventions
Studies included in the review used variety of different
interventions. However, all interventions used either a
computer program (Bloomfield et al., 2010; Cega et al.,
2007; Harrington and Walker, 2004, 2006; Tsai et al., 2004)
or e-learning (Beeckman et al., 2008; Horiuchi et al., 2009;
Hsiao Sheen et al., 2008; Irvine et al., 2007; Mcvey et al.,
2009; Paladino and Peres, 2007). The purpose for inter-
vention programs was to educate nurses and student
nurses using different contents. Learning methods also
varied regarding delivering the course, and in the text and
multimedia methods used, such as plain text, Power-
PointTM (a product by Microsoft1), video, film, graphics,
pictures, or images. All interventions were relatively short
between 40 min to 330 min. One study reported duration
of 60 days for an intervention (Mcvey et al., 2009) (Table 2).
Eight studies out of 11 (Beeckman et al., 2008;
Bloomfield et al., 2010; Cega et al., 2007; Harrington and
Walker, 2004; Horiuchi et al., 2009; Hsiao Sheen et al.,
2008; Paladino and Peres, 2007; Tsai et al., 2004) used
traditional teaching as a comparison for e-learning
intervention. Traditional teaching was mainly done using
Table 1
Characteristics of included studies (N = 11).
Authors Title
Beeckman et al. (2008) Pressure ulcer’s: e-Learning to improve
classification by nurses and nursing
students
Bloomfield et al. (2010) The effect of computer-assisted
learning versus conventional teaching
methods on the acquisition and
retention of hand washing theory and
skills in pre-qualification nursing
students: A randomized controlled trial
Cega et al. (2007) Computer-aided vs. tutor delivered
teaching of exposure therapy for
phobia/panic: Randomised controlled
trial with pre-registration nursing
students
Harrington and The effects of computer-based training
Walker (2004) on immediate and residual learning of
nursing staff
Harrington and Teaching ergonomics to nursing facility
Walker (2006) managers using computer-based
instruction
Methods Participants
Allocation: Randomized, computer Participants: Nursing students (final
generated sequences year), qualified nurses
Blindness: No N: 426, nurses 212, students 214
Design: Parallel Age: 25–44
Location: Multicenter Gender: Female
Setting: Private computer classroom,
traditional classroom
Excluded: No further detail
Allocation: Randomized, computer Participants: Nursing students (first
generated sequences year)
Blindness: Assignment concealed until N: 231
after all participants had been recruited Age: 17–15
Design: Parallel Gender: Female n = 193, Male n = 38
Location: Single located Setting: Individual computer terminal
in an on-campus computer-room,
clinical skills room
Excluded: No further details
Allocation: Randomized, allocated using Participants: Nursing students (third
computer-generated tables year and second year)
Blindness: No N: 92
Design: Parallel, cross over Age: Mixed age
Location: Single located Gender: Mixed gender
Setting: Individual computer terminal
in a classroom, traditional classroom
Excluded: No further details
Allocation: Random assignment Participants: Staff of nursing facilities
Blindness: No N: 1294
Design: Parallel Age: Mean age 44
Location: Multicentre Gender: Female n = 1109, Male n = 185
Setting: Nursing facilities
Excluded: No further details
Allocation: Random assignment Participants: Nursing facility managers
Blindness: No N: 45
Design: Parallel Age: Mean age 45
Location: Multicentre Gender: Female n = 33, Male n = 12
Setting: Nursing facilities
Excluded: No further details
Outcome
Data not able to use:
1. Classification skills for pressure
ulcers
Data not able to use:
M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
1. Knowledge gain,
2. Skill performance checklist
1. Knowledge gain (MCQ1, MCQ2)
2. Skills application (two cases
scenarios)
Data not able to use:
1. Knowledge gain – Knowledge on fire
safety
2. Attitudes towards fire safety
3. Prevention practices related to fire
safety
4. Usefulness of the program, content
and format
1. Knowledge gain – staff knowledge
related to ergonomics
2. Staff attitudes related to ergonomics
3. Staff practices related to ergonomics
Data not able to use
4. Usefulness of the program
5. Effectiveness of the user-interface
141
 142
Table 1 (Continued )

Authors Title Methods Participants Outcome

Horiuchi et al. (2009) Evaluation of a web-based graduate
continuing education program in
Japan: a randomized controlled trial
Allocation: Randomized, computer Participants: Registered nurses or 1. Knowledge gain – knowledge test
generated sequences midwifes 2. Course evaluation Data not able to
Blindness: No N: 70 use
Design: Parallel Age: Mean age 34 3. Satisfaction with learning modality
Location: Multicentre Gender: Female n = 69, Male n = 1
Setting: Web-based learning access on
the course at home or workplace, face-
to-face lectures in nursing school
Excluded: No further details

Hsiao Sheen et al. (2008) e-Learning education programme for Allocation: Randomly allocated by Participants: Registered nurses Data not able to use:
registered nurses: the experience of a flipping a coin N: 42 1. Nursing care skill test

M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149

teaching medical center
Blindness: No Age: Mean age 31 2. Attitudes towards the programmed
Design: Parallel Gender: Females 42
Location: Single located Setting: Intervention in hospital
intranet, in computer class, comparison
no further details
Excluded: Nurses who did not complete
either ELEP or TICP during the study

Irvine et al. (2008) Internet training for nurse aides to
prevent resident aggression
Allocation: Random assignment Participants: Professional
Blindness: No caregivers = nurse aides
Design: Parallel N: 72, 62 completed
Location: Single located Age: Over 18
Gender: Female n = 53, Male n = 9
Setting: Internet
Excluded: No further detail
1. Knowledge gain – VST responses to
videos of aggressive situations
2. Self-efficacy
3. Attitude
4. Behavioral intention

Mcvey et al. (2009) A controlled evaluation of web-based
training for teachers and public health
practitioners on the prevention of
eating disorder
Allocation: Randomly allocated by coin Participants: Elementary school 1. Knowledge gain
Blindness: No teachers and public health practitioners 2. Efficacy to fight weight bias Data not
Design: Parallel N: 165 (teachers 78, health able to use
Location: Multicentre practitioners 89) 3. Computer and internet use
Age: Not described 4. Satisfaction
Gender: Female n = 147, Male n = 30
Setting: Not described
Excluded: No further details

Paladino and Peres (2007) e-Learning: A comparative study for
knowledge apprehension among
nurses
Allocation: Random assignment Participants: Registered nurses 1. Knowledge gain – Knowledge test
Blindness: No N: 49 about quality tool
Design: Parallel Age: Not described
Location: Single located Gender: Not described
Setting: Nursing management rooms
with computers, hospital training room
Excluded: No further details

Tsai et al. (2004) Evaluation of computer-assisted
multimedia instruction in intravenous
injection
Allocation: Random assignment Participants: First year novice nurses
Blindness: No N: 81
Design: Parallel Age: Mean age 23
Location: Single located Gender: Female n = 78, Male n = 3
Setting: In hospital facilities
Excluded: No further details
1. Knowledge gain – Intravenous
injection knowledge test
2. Self-perceived intravenous
performance questionnaire Data not
able to use
3. Satisfaction questionnaire for e-
learning
 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
contact teaching such as face-to-face lecturing and instruc-
tor led modules. The time for control group education varied
between 60 min and 360 min and it took slightly longer than
intervention group education (Table 2).
4.5. Quantitative data synthesis
4.5.1. Impact of e-learning on knowledge level
Eleven studies reported the outcomes of e-learning or
computer assisted learning on participants’ knowledge.
Eight studies (Beeckman et al., 2008; Bloomfield et al.,
2010; Cega et al., 2007; Harrington and Walker, 2004;
Horiuchi et al., 2009; Hsiao Sheen et al., 2008; Paladino and
Peres, 2007; Tsai et al., 2004) compared e-learning to
traditional learning. Four of these studies (Cega et al., 2007;
Horiuchi et al., 2009; Paladino and Peres, 2007; Tsai et al.,
2004) were usable for meta-analysis; three studies had to
be excluded from the quantitative meta-analysis because
the data were unusable due to lack of numerical data
(Beeckman et al., 2008; Bloomfield et al., 2010; Harrington
and Walker, 2004).
Meta-analysis of studies comparing e-learning to
traditional learning (Cega et al., 2007; Horiuchi et al.,
2009; Paladino and Peres, 2007; Tsai et al., 2004) showed
slight impact on e-learning and heterogeneity I2 = 27%,
meaning that these interventions were suitable for meta-
analysis. Although the random effect size showed some
improvement associated with e-learning compared to
traditional techniques, this was not statistically significant
(p = 0.39, MD 0.44, 95% CI 0.57 to 1.46) (Fig. 3).
4.5.2. Impact of e-learning on skills
Four studies reported the outcomes of e-learning or
computer assisted learning on participants’ skills level
(Beeckman et al., 2008; Bloomfield et al., 2010; Cega et al.,
2007; Hsiao Sheen et al., 2008). However, only one study
(Cega et al., 2007) reported the data in a form suitable for
meta-analysis. It showed a slight impact on e-learning.
However, this was not statistically significant (p = 0.13, MD
0.03, 95% CI 0.09 to 0.69). Three studies (Beeckman et al.,
2008; Bloomfield et al., 2010; Hsiao Sheen et al., 2008)
reported the data in such a way that it was not usable due
to a lack of numerical data.
4.5.3. Impact of e-learning on satisfaction
Three studies (Horiuchi et al., 2009; Mcvey et al., 2009;
Tsai et al., 2004) reported participant satisfaction with
e-Learning Traditional learning
Study or Subgroup Mean SD Total Mean SD
Cega 2007 71 10 83 73 10
Horiuchi 2009 76.2 18.8 37 79.4 21.5
Paladino 2007 1.7 2.8 24 1.3 3.8
Tsai 2004 9.67 0.53 42 8.82 0.6
Total (95% CI) 186
Heterogeneity: Tau² = 0.37; Chi² = 4.14, df = 3 (P = 0.25); I² = 27%
Test for overall effect: Z = 0.86 (P = 0.39)
Fig. 3. Forest plot of e-learning vs. traditional learning.
143
learning methods. However, the data from these three
studies were not usable due to lack of numerical data.
5. Discussion
This systematic review aimed to investigate the impact
of e-learning on knowledge, skills and satisfaction among
nurses and nursing students compared to traditional
education methods. The preliminary evidence suggests
that individualized, tailored e-learning approaches are
more effective than traditional interventions (Lancaster
and Stead, 2005). This finding gave a promising lead in
health education and also guides the assumptions in our
study. Keeping in this finding in mind we found only 11
studies out of 177 reporting effectiveness of e-learning on
participants’ knowledge level. The finding is surprising,
although e-learning is one of the fastest growing methods
in the education industry (European Commission, 2010;
Digital Agenda Assembly, 2011). As has been found in an
earlier review (e.g. Cook et al., 2008) most of the reports
described comparisons with no intervention and had
therefore to be excluded from the analysis.
The results of this review showed that e-learning is not a
superior learning method to traditional learning methods.
Concerning our primary outcome, i.e. knowledge level, no
statistical differences between learning outcomes were
found. The finding is in line with previous reviews (Curran
and Fleet, 2005; Wutoh et al., 2004), although more positive
results can also be found in earlier reviews (e.g. Cook et al.,
2008). The reasons for this may for example have to do with
the method itself or the content of the education
investigated in the studies included. In general, interactivity,
practice exercises, repetition, and feedback improved
learning outcomes (Cook et al., 2010a) and e-learning offers
a flexible method to include these elements in education.
However, we do not have detailed information on how those
elements were used in the interventions under investiga-
tion. Furthermore, there is also a need to explore more
carefully possible content related limitations on the use of e-
learning. In other words, whether e-learning is a more
relevant method to provide education on certain topics.
We were also interested in whether e-learning is more
effective in improving learners’ skills than traditional
methods. Our analysis yielded no evidence for our
assumption, while no statistical differences between e-
learning and traditional learning methods to improve
participant’s skills were found. To improve learners’ skills
Mean Difference Mean Difference
Total Weight IV, Random, 95% CI IV, Random, 95% CI
78 9.3% -2.00 [-5.09, 1.09]
33 1.1% -3.20 [-12.71, 6.31]
25 20.9% 0.40 [-1.46, 2.26]
39 68.6% 0.85 [0.60, 1.10]
175 100.0% 0.44 [-0.57, 1.46]
-4 -2 0 2 4
Favours traditional learn Favours e-learning

Table 2
Description of intervention and control learning methods.
Authors
Beeckman et al. (2008)
Bloomfield et al. (2010)
Cega et al. (2007)
Harrington and Walker (2004)
144
Method Description of intervention Time of Method Description of control Time of control
intervention
e-Learning Name: The Pressure Ulcer 60 min Lecture Name: Lecture related to 60 min
Classification (PUCLAS) pressure ulcers classification
Topic: Programme provides Topic: Lecture provides same
information about pressure content than e-Learning
ulcers and its classification programme
Phases: Exercises on different Phases: –
levels of difficulty Methods: Lecture using
Methods: Text, photographs PowerPoint
and video Time: 1 h
M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
Time: 1 h Self-standing: –
Self-standing: Yes
Computer programme Name: – 90 min Lecture Name: – 90 min
Topic: Programme related hand Topic: Lecture related hand
washing technique washing technique
Phases: Six steps for right hand Phases: Lecture and practice
washing Methods: Lecture, slides, video,
Methods: Text, interactive additional material and
activities: animated practice session
multimedia, photographs, Time: 90 min
video and links to website Self-standing: –
Time: 90 min
Self-standing: Yes
Compuer programme Name: ‘‘FearFighter’’ 60 min Lecture Name: – 60 min
Topic: Programme explains Topic: Experienced lecturer
how exposure therapy works gave a powerpoint presentation
with phobia/panic covering content like
Phases: 9 steps ‘‘FearFighter’’
Methods: – Phases: Lecture
Time: Each step takes 5–20 min Methods: Lecture using
Self-standing: Yes PowerPoints, discussion and
answering questions
Time: 60 min
Self-standing: –
Computer program Name: ‘‘Fire Safety in Nursing NA Instructor led Name: – NA
Facilities’’ modules Topic: Includes material related
Topic: Programme includes fire safety in nursing facilities
material related fire safety in Phases: 6 modules
nursing facilities Methods: Instructor led
Phases: 6 modules modules
Methods: Storyboard, videos, Time: One module, 34–43 min
questions, games, text and Self-standing: –
animation.
Time: One module, 26–41 min
Self-standing: Yes
Harrington and Walker (2006)
Computer programme Name: ‘‘Ergonomics for Nursing NA – No training NA
Facility Mangers’’
Topics: Programme includes
information related to
ergonomics. Six main topics.
Phases: Programme have 54
separate screen and 44 photos
and also interactive exercises.
Methods: Text, graphics,
photographs, animation,
sounds, interactive questioning
and answering.
Time: NA
Self-standing: Yes

M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149

Horiuchi et al. (2009)
e-Learning Name: ‘‘Evidence-Based 120 min, in 1 Lecture Name: – 360 min in 1 month period
Nursing’’ month period Topics: 4 lectures related to
Topics: Programme includes EBN
information related to EBN Phases: 4 parts
Phases: 4 parts Methods: Face-to-face lecture
Methods: NA Time: Each part took 90 min.
Time: Each part took 30 min. Programme completed in 1
Programme completed in 1 month.
month. Self-standing: –
Self-standing: Yes

Hsiao Sheen et al. (2008)
e-Learning Name: ‘‘ELEP’’
Topics: Programme included
case study, career
development, teaching and
learning, nursing and law and
communication
Phases: 5 phases
Methods: Audio, video and
PowerPoints.
Time: 330 min
Self-standing: Yes
330 min, in a 3 Lecture Name: ‘‘TICP’’ 330 min, in a 3 month
month period. Topics: Lectures included case period
study, career development,
teaching and learning, nursing
and law and communication
Phases: 5 lectures
Methods: Presentations
Time: 330 min
Self-standing: –

Irvine et al. (2008) e-Learning Name: ‘‘Professional dementia NA – No training NA

care: Managing Aggression’’
Topics: Programme included
information related to
approaching agitated resident
and safety de-escalating
techniques.
Phases: NA
Methods: Written text,
graphics, images, testimonials,
videos and voice over clips
Time: NA
Self-standing: Yes

145
 146
Table 2 (Continued )

Authors Method Description of intervention Time of Method Description of control Time of control
intervention

Mcvey et al. (2009) e-Learning Name: ‘‘The Student Body: 60 Days access to – No training NA
Promoting Health at Any Size’’ online programme
Topics: Programme included
information related to health at

M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149

any size
Phases: Programme includes
six modules and each module
consist four steps
Methods: Written text,
animation, videos and links
Time: NA
Self-standing: Yes

Paladino and Peres (2007) e-Learning Name: ‘‘Quality tools’’ 40 min Lecture Name: – 120 min
Topics: Programme included Topics: Programme included
information related to hospital information related to hospital
quality quality
Phases: NA Phases: NA
Methods: Programme used Methods: Lecture used
powerpoint presentations powerpoint presentations
Time: 40 min Time: 120 min
Self-standing: Yes Self-standing: –

Tsai et al. (2004) Computer programme Name: ‘‘Intravenous injection’’ 90 min Lecture Name: – NA
Topics: Programme included Topics: Orientation lecture
information related to included information related to
intravenous injection and its intravenous injection and its
administration administration
Phases: Three 30 min sessions Phases: –
Methods: Programme Methods: Lecture
delivered via CD-ROM, Time: NA
programme using text, voice, Self-standing: –
film and pictures
Time: 120 min
Self-standing: Yes
 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
there is a need to integrate the content of the education
into clinical nursing practice (Morgan, 2006). On the other
hand, e-learning methods are still fairly new in this field
and this aspect may not be considered in existing
educational interventions. Therefore it may be that
existing educational interventions cannot adequately
respond to participants’ needs and experiences in clinical
practice. Thus, to improve participants’ skills by using e-
learning this integration should be taken into account in
the development of e-learning interventions (Murdoch
et al., 2012).
Lastly, we assumed that e-learning has a positive
impact on learners’ satisfaction with education. However,
from our data it was not possible to find answers to our
question due to missing statistical information. Earlier
studies (e.g. Wutoh et al., 2004) have reported participants’
satisfaction related to educational methods showing better
outcome for e-learning. In our review of qualitative
analysis on satisfaction Tsai et al. (2004) showed that
students are more satisfied with e-learning. Also, Mcvey
et al. (2009) showed similar effects on satisfaction in their
study on the use of e-learning (also Cook et al., 2008;
Curran and Fleet, 2005). Whether this is true or not, more
high quality RCT studies should be conducted.
5.1. Strengths and limitations of the review
This review has both strengths and limitation. First, the
strengths of this review include a systematic approach to
searching, screening and reviewing studies and extracting
data using standardized forms and duplicating all stages.
Second, the search area was also large and the inclusion
criteria broad enough to encompass the broadest range of
interactive e-learning interventions.
The review also has some limitations. Although every
effort was made to locate published trials our findings may
still include selective reporting. The study also involves
judgments made by the review authors, either of which
may lead to bias. Also, the outcome data was sparsely
reported and due to this we found that a large number of
the papers included did not report any numerical data. This
may cast doubt on the validity of the inclusion procedures.
All studies included in this review were randomized.
However, the details were not frequently provided. It is
therefore unclear in almost all studies whether randomi-
zation was really appropriately done. The interventions
were also rather heterogeneous which might raise the
question about the meaningful use of the meta-analysis.
However, all interventions used e-learning as a interven-
tion and therefore the meta-analysis was considered useful
and suitable method (Higgins and Green, 2011). In
addition, the research papers in this review used rather
small samples and adequate power calculations to
estimate sample sizes were not presented in the papers.
Only two studies (Cega et al., 2007; Paladino and Peres,
2007) showed that their studies were powered.
Moreover we found considerable limitations regarding
the tools for measuring study outcomes. In general they
were only briefly described in the studies, so no appraisal
of measurement tools can be presented in this review.
Especially regarding the measurements of knowledge it is
147
important to understand that assessing knowledge can
take many forms with a number of different conceptual
definitions. Therefore, one might question whether the
data can be combined without manipulation, especially
when all the studies included used self-developed instru-
ments. Moreover, overall results are much influenced by
the Tsai et al. (2004) study due to its small standard
deviation. However, we considered the scales similar
enough to be combined in the analysis. The same approach
was used in a systematic review by Cook et al. (2008). Due
to these limitations there is a risk of overestimating the
effects of the experimental interventions. Therefore there
is still a need for further studies to assess the impacts of e-
learning in nursing education.
5.2. Implications for practice
Our review can inform educational policymakers and
healthcare commissioners in deciding whether e-learning
should be part of the nursing curriculum in basic education
or on-the-job training in nursing education. According to
our analysis none of the learning methods used, neither e-
learning nor more traditional learning methods are
superior to increase students’ knowledge level, skills, or
satisfaction with the learning methods. However, the
knowledge base is not strong due to the lack of high quality
research. It is therefore important to be aware that
different educational methods including e-learning are
available as means of educating nurses and nursing
students.
Based on the results of this review we can agree with
Cook et al. (2010a,b) who concluded in their review the
educators and researchers cannot treat web-based learn-
ing methods as a single entity. Educators should carefully
consider the needs of their learners and the learning
context in question when deciding whether to use this
medium, other media or a combination. Those electing to
use e-learning should consider a broad range of methods
before choosing a specific approach. Instead of concluding
whether e-learning methods are good or bad based on the
authors’ best understanding, these methods should be
used with full awareness of participant groups and their
needs.
5.3. Implications for research
This review showed clearly the need to need to conduct
studies in the field of e-learning. The evaluation of
educational interventions should focus on a variety of
outcomes as noted in Kirkpatrick’s model of learning
outcome evaluation (Galloway, 2005). Future studies
should use adequate power calculations to be properly
weight. Also, the randomization process should be
conducted and reported in greater detail so that sufficient
validity assessment is possible. Moreover, researchers
need to pay attention to the reporting the numerical data.
In general, the importance of the CONSORT and QUOROM
statements should be taken more seriously into account in
future studies. Following these recommendations would
clearly improve the conduct and reporting of empirical
research on certain topic following a methodological
148 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
protocol such as randomized controlled trials. These
recommendations would also facilitate critical appraisal.
The QUOROM statement and flow chart of the review aim
to produce better quality reviews and provide evidence-
based knowledge for practitioners (Turpin, 2005).
6. Conclusion
This systematic review of the impact of e-learning
compared to traditional education in the acquisition of
knowledge, skills and satisfaction among nurses and
student nurses’ showed that there is no difference between
e-learning or traditional learning. Even the e-learning is
not superior to traditional learning it can, however, offer an
alternative method of education. In general, this review
offers important information to increase knowledge base
about effectiveness of different education methods. This
information can be used to support decision makers to
choose those methods which have impacts on students’
satisfaction and knowledge level as well support change in
practice. In future there is still need to develop and
evaluate methods for education among nurses. It is
important that these studies are conducted to show
effectiveness and they should also focus on the variety
of different levels of outcomes.
Conflict of interest: None declared.
Funding: Funding received for this study from Turku Uni-
versity Foundation and The Finnish Association of Nursing
Research. Funders did not have any role in the study.
Ethical approval: No ethical permission was needed.
References
Ades, A., Higgins, J., 2005. The interpretation of random-effects meta-
analysis in decision models. Medical Decision Making 25, 646–654.
Beeckman, D., Scoonhoven, L., Boucque, H., Van Maele, G., Defloor, T.,
2008. Pressure ulcers – e-learning to improve classification by nurses
and nursing students. Journal of Clinical Nursing 17, 1697–1707.
Bloomfield, J., Roberts, J., While, A., 2010. The effects of computer assisted
learning versus conventional methods on the acquisition and reten-
tion of handwashing theory and skills in pre-qualification nursing
students: a randomized controlled trial. International Journal of
Nursing Studies 47, 287–294.
Belcher, J., Vonderhaar, K., 2005. Web-delivered research-based staff
education for seeking magnet status. Journal of Nursing Administra-
tion 35 (9), 382–386.
Cega, L., Norman, I., Marks, I., 2007. Computer-aided vs. tutorial-delivered
teahing of exposure therapy for phobia/panic: randomized controlled
trial with pre-registration nursing students. International Journal of
Nursing Studies 44, 397–405.
Cohn, L.D., Becker, B.J., 2003. How metaanalysis increases statistical
power. Psychological Methods 8, 243–253.
Cook, D., Levinson, A., Garside, S., 2008. Internet-based learning in the
health professions. Journal of American Medical Association 30 (10),
1181–1196.
Cook, D., Levinson, A., Garside, S., 2010a. Instructional design variations in
internet-based learning for health profession education: a systematic
review and meta-analysis. Academic Medicine 85 (5), 909–922.
Cook, D., Garside, S., Levinson, A., Dupras, D., Montori, V., 2010b. What do
we mean by web-based learning? A systematic review of the varia-
bility of interventions. Medical Education 44, 765–774.
Commission of the European Communities, 2001. Communication from
the Commission to the Council and the European Parliament. The
eLearning Action Plan. Designing tomorrow’s education. Brussels,
28.3.2001. COM (2001)172 final.
Commission of the European Communities, 2000. e-Learning – Designing
tomorrow’s education. Communication from the Commission. Brus-
sels, 24.5.2000. COM (2000) 318 final.
Commission of the European Communities, 2003. e-Learning – Designing
tomorrow’s education. Commission staff working paper eLearning:
designing tomorrow’s Education A Mid-Term Report As requested by
the Council Resolution of 13 July 2001. Brussels, 30.7.2003 SEC (2003)
905.
Cobb, S., 2004. Internet continuing education for health care profes-
sionals: an integrative review. The Journal of Continuing Education
in the Health Professionals 24, 171–180.
Curran, V., Fleet, L., 2005. A review of evaluation outcomes of web-based
continuing medical education. Medical Education 39, 561–567.
Curran, V., Fleet, L., Kirby, F., 2010. A comparative evaluation of the effect
of internet-based CME delivery format on satisfaction, knowledge and
confidence. Medical Education 10 (10), 1–7.
Digital Agenda Assembly, 2011. Report from the workshop 08. ‘‘Main-
streaming e-Learning in education and training’’ DAE action 6. Brus-
sels, 16–17.7.2011.
Egger, M., Davey-Smith, G., Schneider, M., Minder, C., 1997. Bias in meta-
analysis detected by a simple graphical test. BMJ 315, 629–634.
European Commission, 2010. Communication from the Commission Eur-
ope 2020. A strategy for smart, sustainable and inclusive growth.
Brussels, 3.3.2010. COM (2010) 2020 final.
European Commission, 2011. Communication from the Commission to
the European parliament, the Council, the European Economic and
social committee and committee of the regions. Supporting growth
and jobs – an agenda for the modernisation of Europe’s higher
education systems. Brussels, 20.9.2011. COM (2011) 567 final.
Eurostat, 2011. Healthcare staff statistics at regional level – Statistics
Explained (2011/11/0), http://epp.eurostat.ec.europa.eu/statistic-
s_explained/index.php/Healthcare_staff_statistics_at_regional_level.
Galloway, D., 2005. Evaluating distance delivery and e-learning. Perfor-
mance Improvement 44 (4), 21–27.
Guyatt, G.H., Oxman, A.D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P.,
Montori, V., Akl, E.A., Djulbegovic, B., Falck-Ytter, Y., Norris, S.L.,
Williams Jr., J.W., Atkins, D., Meerpohl, J., Schünemann, H.J., 2011.
GRADE guidelines. 4. Rating the quality of evidence – study limita-
tions (risk of bias). Journal of Clinical Epidemiology 64 (4), 407–415.
Harrington, S., Walker, B., 2004. The effects of computer-based training on
immediate and residual learning of nursing facility staff. The Journal
of Continuing Education in Nursing 35 (4), 154–163.
Harrington, S., Walker, B., 2006. Teaching ergonomics to Nursing facility
managers using computer-based instruction. Journal for Nurses in
Staff Development 22 (5), 260–268.
Higgins, J.P.T., Green, S. (Eds.), 2011. Cochrane Handbook for Systematic
Reviews of Interventions Version 5.1.0. The Cochrane Collaboration
Available at: www.cochrane-handbook.org (retrieved 24.1.2013).
Higgins, J., Thompson, S., 2002. Quantifying heterogeneity in meta-ana-
lysis. Statistics in Medicine 21, 1539–1558.
Higgins, J., Thompson, S., Deeks, J., Altman, D., 2003. Measuring incon-
sistency in meta-analyses. British Medical Journal 327, 557–560.
Horiuchi, S., Yaju, Y., Koyo, M., Sakyo, Y., Nakayama, K., 2009. Evaluating of
a web-based graduate continuing nursing education program in
Japan: a randomized controlled trial. Nurse Education Today 29,
140–149.
Hsiao Sheen, S.-T., Chang, W.-Y., Chen, H.-L., Chao, H.-L., Tseng, C., 2008. E-
learning education programme for registered nurses: the experience
of a teaching medical center. Journal of Nursing Research 16 (3), 195–
200.
Irvine, B., Bourgeois, M., Billow, M., Seeley, J., 2007. Internet training for
nurse aides to prevent resident aggression. Journal of the American
Medical Directors Association 8 (8), 519–552.
Khan, K., Kunz, R., Kleijnen, J., Antes, G., 2003. Systematic Reviews to
Support Evidence-Based Medicine. How to Review and Apply findings
of Health Care Research. RSM Press, London.
Kirkpatrick, D., 1998. Evaluating Training Programs. The Four Levels.
Berrett-Koehler Publishers, San Francisco.
Lancaster, T., Stead, L., 2005. Self-help interventions for smoking cessa-
tion. Cochrane Database of Systematic Reviews (3), http://dx.doi.org/
10.1002/14651858.CD001118.pub2 Art. No.: CD001118.
Leucht, S., Engel, R.R., Bauml, J., Davis, J., 2007. Is the superior efficacy of
new generation antipschotics an artifact of LOCCF? Schizophrenia
Bulletin 33 (1), 183–191.
Lowry, M., Johnson, M., 1999. Computer assisted learning: the potential
for teaching and assessing in nursing. Nurse Education Today 19,
521–526.
Mcvey, G., Gusella, J., Tweed, S., Ferrari, M., 2009. A controlled evaluation
of web-based training for teachers and public health practitioners on
prevention of eating disorders. Eating Disorders 17, 1–26.
 M. Lahti et al. / International Journal of Nursing Studies 51 (2014) 136–149
Moher, D., Liberati, A., Tetzlaff, J., Altman, D., The PRISMA Group, 2009.
Preferred reporting items for systematic reviews and meta-analyses:
the PRISMA statement. PLoS Medicine 6 (6), e1000097 http://
dx.doi.org/10.1371/journal.pmed1000097 (retrieved 24.1.2013).
Morgan, R., 2006. Using clinical skills laboratories to promote theory-
practice integration during first practice placement: an Irish perspec-
tive. Journal of Clinical Nursing 15 (2), 155–161.
Moore, D., McCabe, G., 2003. Introduction to the Practice of Statistics, 4th
ed. W.H. Freeman & Company.
Murdoch, D., Bushell, C., Johnson, S., 2012. Designing simulations for
professional skill development in distance education: a holistic
approach for blended learning. In: Holt, D., Segrave, S., Cybulski, J.
(Eds.), Professional Education Using E-Simulations: Benefits of Blended
Learning Design. IGI Global, (10 January 2012), pp. 121–140.
Paladino, Y., Peres, H., 2007. E-learning: a comparative study for knowl-
edge apprehension among nurses. The Revista Latino-Americana de
Enfermagem 15 (3), 397–403.
Ray, J., Shadish, W., 1996. How interchangeable are different estimators of
effect size? Journal of Consulting and Clinical Psychology 64 (6),
1316–1325.
Savoie, I., Helmer, D., Green, C., Kazanjian, A., 2003. Beyond Medline:
reducing bias through extended systematic review search. Interna-
tional Journal of Technology Assessment in Health Care 19 (1),
168–178.
149
Schittek, M., Mattheos, N., Lyon, H., Attström, R., 2001. Computer assisted
learning. A review. European Journal of Dental Education 5, 93–100.
Simpson, B., 2003. Web-based and computer-assisted instruction in physical
therapy education. Journal of Physical Therapy Education 17, 45–49.
Smith, C., 2005. E-orientation: a cyber approach to orienting per diem and
temporary nurses. Journal for Nurses in Staff Development 21 (5),
204–212.
Tsai, S.-L., Tsai, W.-W., Chai, S.-K., Sung, W.-H., Doong, J.-L., Fung, C.-P.,
2004. Evaluation of computer assisted multimedia instruction in
intravenous injection. International Journal of Nursing Studies 41,
191–198.
Turpin, D., 2005. CONSORT and QUOROM guidelines for reporting ran-
domised clinical trials and systematic reviews. American Journal of
Orthodontics and Dentofacial Orthopedics 128 (6), 681–685.
University of York, 2008. Systematic Reviews: CRD’s Guidance for Under-
taking Reviews in Health Care. University of York, Centre for Reviews
and DisseminationIn: http://www.york.ac.uk/inst/crd/pdf/Systema-
tic_Reviews.pdf (retrieved 9.1.2012).
Ward, R., Stevens, C., Brentnall, P., Briddon, J., 2008. The attitudes of health
care staff to information technology: a comprehensive review of the
literature. Health Information and Libraries Journal 25, 81–97.
Wutoh, R., Boren, S., Balas, A., 2004. eLearning: a review of internet-based
continuing medica education. The Journal of Continuing Education in
the Health Professions 24, 20–30.