Using Leadership Simulation Scenarios

With Graduate Nursing Learners to

Support Frontline Nurse Leader Competency
Development
Linda Cole, DNP, RN, APRN, CCNS, CNE, CPHQ,
Lisa Boss, PhD, RN, APRN. CNS, CEN CNE,
Debra Fowler, PhD, MBA, RN, CNE, NEA-BC, BCC, and
Latarsha Cheatham, DNP, RN-BC, APRN, FNP-BC

Although the use of simulation has grown in education, there is little written about how it can
be used with frontline nurse leaders (FLNLs) to teach nontechnical skills such as critical
thinking, teamwork, communication, and conflict resolution. This article discusses the
implementation of FLNL simulation in a graduate nurse leadership program based on the
American Organization for Nursing Leadership (AONL) Nurse Manager Competencies. Learner
confidence increased in the AONL competencies post-simulation. Although the setting of this
project was the academic setting, the approach described could be easily adapted to the
clinical setting for orientation and education of FLNLs

he American Organization for Nursing

T
to better develop these skills. As a result, frontline
Leadership (AONL)1 developed competencies nurse leader (FLNL) simulation was developed and
for nurse leaders to use as standards in a proposed to support the learner’s development of
variety
key nontechnical skills. The purpose of this article
of care settings that include acute care, post-acute
is to describe the process used and propose how a
care, and system-level positions. Faculty of the
similar process could be implemented in a practice
Master of Science in Nursing Leadership (MSN in
setting to develop frontline leaders.
NL) program at Cizik School of Nursing use the
BACKGROUND
AONL Nurse Manager Competencies1 as a
Nurse educators have successfully
framework for the curriculum, which is focused on
incorporated simulation into nursing
developing frontline nurse leaders. In the MSN in NL
education to provide learners a safe
program, enrolled students include nurses who are
environment to practice high-risk and/or
in both formal and informal leadership roles.
high acuity skills in both undergraduate and
Feedback from program alumni as well as senior
graduate education.2 Most of these
nursing leaders in the area indicated frontline nurse
simulations are focused on clinical
leaders must be proficient in nontechnical skills
management and utilize manikins or
such as critical thinking, teamwork,
standardized patients in the clinical setting.2
communication, and conflict resolution. Although
Although simulation is well-documented for
course faculty have implemented various teaching
its success in teaching clinical
approaches to build core competencies, the faculty
determined that additional, active learning
approaches could be used

There is limited nursing literature on the use of simulation with frontline
Organization for Nursing Leadership Nurse Manager Competencies increased learners’ confidence levels. Leadership
simulations could be easily adapted to the
clinical setting for orientation and
education of FLNLs.
skills, little is known about the use of
simulation for teaching nontechnical skills,
such as leadership skills.
Researchers have studied simulation to
teach “basic” leadership skills such as
communication, teamwork, delegation, and
priority setting; however, the context is
limited to the clinical setting or “at the
bedside.”3 This means the leadership skills are
specific to scenarios where patients are
directly involved, and the goal is to establish
a basic level of communication for the
bedside nurse. Notably, a lack of published
studies exist that aim to investigate the use of
simulation as a teaching approach to develop
advanced leadership skills for the frontline
nursing leader, such as transformational
leadership skills, provision of discipline, and
negotiation of disputes. 4 objectives of the course.
To best prepare MSN in NL graduates to
effectively function in a complex, rapidly
changing health care system, nursing faculty
must create an innovative and active learning
environment where learners develop
5
advanced leadership skills. The gap in
published evidence related to the use of
simulation to teach advanced leadership skills
provides an ideal opportunity for nursing
faculty to create an active learning activity
via the use of SBL while contributing
meaningful evidence to the body of science.
Framework
Constructivism learning theory provided a
theoretical framework for this project. In
constructivism, teaching and learning are
based on the premise that learners learn best
KEY POINTS
by fitting new information together with prior
knowledge.
The theory
nurse leaders (FLNLs)suggests
to teach that learners skills such as critical
nontechnical
construct knowledge and meaning from
their
experiences, and learning is an active
process.6 Additionally, adult learning theory
asserts that adults will learn information that
is useful and relevant to them and are actively
involved in the learning process.7 Simulation
is a well- documented example of active
learning that incorporates adult learning
theory.
The principles from the book Crucial
Conversations8 served as a secondary
framework for this project. The book outlines
specific steps that can be employed to
effectively hold conversations in situations
that are have high stakes, varying opinions,
and/or strong emotions. Frontline nurse
leaders are faced with these types of
conversations on a daily basis and must
demonstrate mastery of these skills to
effectively lead and achieve organizational
outcomes.
Specific Aim of the Project
The aim of this quality improvement (QI)
project was to evaluate the effectiveness of
using FLNL simulation scenarios to increase
learner’s confidence in their competency of
AONL Nurse Manager Competencies1 related
to relationship management and influencing
behaviors. These particular competencies
were selected as they aligned with the
METHODS Interventions
With faculty oversight, FLNL simulation scenarios
were developed by senior-level MSN in NL learners
with the intent to create relevant, “real-world”
scenarios a frontline nurse leader would likely
encounter in the clinical setting. Actors for the
scenarios were the same senior-level MSN in NL
learners. Seven scenarios were designed that could
be altered from “beginner” to “advanced” level. An
example of 1 of the scenarios used is provided in
Figure 1. Each learner participated in a FLNL
simulation scenario geared to the learner’s level of
leadership experience. The learners had a variety of
leadership experiences ranging from staff nurse
committee chairperson to charge nurse to assistant
nurse manager and shift nurse manager.
Before the FLNL simulation activity, the
learners read Crucial Conversations8 and
participated in an online discussion about the book
and how the
 principles from the book may have changed a the course participated in the in-class, FLNL
previous unsuccessful conversation. All learners in simulation activity. Prior to the FLNL
simulation activity, learners were asked to their simulation session, discuss how the learner felt
complete an anonymous pre-survey, and after the during the simulation activity, and receive feedback
simulation activity, learners were asked to complete from peers, senior MSN in NL learners, and faculty.
the same anonymous survey along with open-ended
qualitative questions. Learners generated their own
unique identifier code so that pre- and post-surveys Measures
could be linked. The pre-simulation survey was composed of
Each FLNL simulation scenario lasted questions focused on the participant’s perceived
approximately 10 minutes. The simulation took confidence before and after FLNL simulation as
place in a room that resembled a typical work related to the AONL Nurse Manager Competencies 1
setting. Faculty and students who were not engaged of relationship management and influencing
in the simulation watched a live video on a large behaviors. This competency can be subsectioned
monitor from a nearby conference room. After each into 5 areas: managing
AONL Manager Competencies Learning Objectives:
FLNL simulation scenario, an in-depth peer-to-peer The Art
A. Human Resource Leadership Skills
debriefing occurred. The goal of debriefing was to 1. Performance Management

provide participants with the opportunity for • Initiate corrective actions

B. Relationship Management and Influencing Behaviors
meaningful reflection and analysis. Faculty guided 1. Situation management

the discussion and posed specific questions, • Identify issues that require immediate attention

including: 1) What went well in the scenario? 2) 2. Influence others

• Role model professional behavior
What could have been done better? 3) What did you Scenario Synopsis

learn by participating in or watching this scenario? Tam has now reached an occurrence (written document) due to being late more than 12 times.
Per policy, being late 6 times equals a verbal warning, 12 times late is a written warning, 16
and 4) Which principle(s) was/were used from times is a final warning, and 20 times late is termination (in 1 calendar year).

Tam is a single mom who is also in college and contemplating dropping school as she is
Crucial Conversations8? During the debriefing, each struggling to keep it all together.

learner had an opportunity to review a recording of Set Up

Setting/Environment: Women’s Center
ER Behavioral Health
Med-Surg Executive Suites
Peds Office
ICU Meeting
OR / PACU Other:
Room staging: Setting is in an office between manager and Tardy Tam
Additional props: n/a
Character names and roles: Tam (late nurse) and Student–Manager
Pre-brief: policy states: being late 6 times equals a verbal, 12 times late is a written warning, 16 times is a final warning, and 20 times late is termination (in 1 calendar year). Being late is indicated at 10 minutes past the scheduled time of duty.

Nurse Tam is a high performer on your unit and always positive. However, she is frequently late. Tam received a verbal warning for lateness 2 months ago but has since accrued 6 more and is due for a formal written warning. The written
discipline means she will lose her bonus for the quarter and cannot transfer from the department for 6 months. You need to deliver the document and have her sign it.

Optional challenges for learners with some leadership experience: Tam is not a high performer as she does the bare minimum. She has a poor attitude and has told the team she does not need this job. The team talks about how Tam is always
late, and there are rumors that she is friends with the secretary who will correct her late timecard. She has accrued enough for a final warning document. Four more times late and she will be terminated. Deliver the document.

Scenario Progression Outline

Timing Patient/Staff Actions Expected Leader Interventions May Use the Following
(approx.) Cues

0–2 min Jokes about going to principal’s office Opening conversation to keep RN comfortable but build Role member providing
report cue: N/A
Cue: N/A

2–4 min Denial Display facts Role member providing cue: N/A
Cue: N/A

4–6 min Anger Remain calm, understanding, don’t take it personally, Role member providing cue: N/A Cue: N/A
but have boundaries for behavior

6–8 min Bargaining/crying/acceptance Coaching, mentoring, offer hope and reassurance/support Role member providing cue: N/A Cue: N/A
 ER, emergency room; ICU, intensive care; OR, operating room; PACU, post-anesthesia care unit.

Figure 1. Sample Scenario. Ethical Considerations

Approval for this project was obtained from
conflict, situation management, relationship the university’s institutional review board
management, influencing others, and promoting prior to data collection. The project
professional development. Each learner rated manager, a faculty member who was not
perceived confidence using a 5-point Likert scale teaching the course, introduced the project
ranging from 1 (very little confidence) to 5 (very high to the learners, invited them to participate,
confidence). and provided the links for the anonymous
The post-simulation survey used the same 5 online survey.
point, Likert-style, confidence-level survey as well
as openended qualitative questions. The qualitative RESULTS Quantitative Results
questions included in the post-FLNL simulation Eight participants completed both
survey requested information regarding elements of pre/postintervention surveys. Demographic
the core competencies the learner had found to be data are summarized using mean values in
challenging in their academic work to date and what Table 2. Of interest from the demographic
they believed would be most valuable to improve data, 50% of the participants stated they were
their confidence. The qualitative questions are listed currently employed in a leadership role such
in Table 1. as assistant nurse manager or shift nurse
Analysis manager, but 87.5% of the participants
Data were gathered via an anonymous noted they had not received any leadership
online survey and downloaded into a secure, training outside of an academic setting.
password-protected database. A unique As illustrated in Table 3, participants
identifier created by each participant was reported increases in perceived confidence
used to match the pre- and post-surveys. in most AONL competency measures after
Analysis was conducted with deidentified simulation. Due to the small sample size,
data. Demographic data were summarized statistical analyses to compare groups were
using mean values. To evaluate preliminary not performed.
efficacies for outcome measures in these
pilot data, the median differences in pre–
post outcome measures were utilized. Due Qualitative Results
to the small sample size (N = 8), statistical Participants reported the realism of the
analyses to compare groups were not scenarios was essential to improving their
performed. confidence. Scenarios helped them critically
Qualitative data gathered in the open- think in “real time” during a difficult
ended postsurvey items were analyzed to conversation, rather than engaging in
identify the impact of simulation on the discussion of hypothetical situations.
participant’s confidence to handle leadership Learners recognized the scenarios were
situations and develop leadership based on challenging experiences similar
competencies. Qualitative questions also to those in their own work settings. They
focused on the experience of the FLNL also learned a good deal through observing
simulation activity so that faculty could how their colleagues handled the
make changes in future simulations based situations.
on participant feedback. The observations enabled them to identify
alternative leadership behaviors and
communication techniques.

What didn’t you like about the simulation exercise?

What changes would you like to see made to the simulation exercise?
Post-simulation debriefing sessions effectiveness of the learning experience. During the
were identified as essential to the debriefing, both faculty and the senior MSN in NL learners
 shared helpful insights from their own
leadership challenges. The process fostered
supportive and constructive critique of self
and peers, and further promoted meaningful
discussions.
The simulation lab environment, which
had recently undergone extensive
standardized time limit for each scenario
and debrief, and a more formal debriefing
process.
DISCUSSION Summary
It is uncommon for graduate student learners to
participate in simulations that address real-life
dilemmas that may surface when fulfilling frontline
nurse leader job expectations. Therefore,
interventions were developed and implemented to
allow the learners to use prior experience, simulation,
and debriefing to increase confidence in core AONL

renovation, was also identified as supportive

of the learning experience. The video
capabilities allowed participants to view
their own behavior and reflect on the
experience as well as receive feedback
during the debrief.
Although the simulation was initially
uncomfortable, participants unanimously
recommended that faculty continue to
include FLNL in future courses.
Suggestions for improvement included
developing a
Nurse Manager
Competencies.1 Qualitative findings were indicative
of learner benefits. The primary outcome was an
increase in learner confidence. This finding is
similar to clinical simulation results, which suggest
that simulation improves learner confidence.9,10
Notably, to our knowledge, this is one of the few
published QI projects that evaluated the
effectiveness of simulation to increase learner
confidence in a graduate nursing leadership
program.

Pre Post
Relationship Management and Influencing Behaviors Median Median
Managing conflict Question 1: How confident were you in managing 3 4
conflict?
Situation management Question 2: How confident were you in identifying 3 4
issues that require immediate leadership attention?
 Relationship management Question 3: How confident were you in promoting 3 4
team dynamics?
Question 4: How confident were you in mentoring 3 4
and coaching staff and colleagues?
Question 5: How confident were you in applying 3 4
communication strategies?
Influencing others Question 6: How confident were you in encouraging 3 4
participation in professional action?
Question 7: How confident were you in role 4 4
modelling professional behavior?
Question 8: How confident were you in acting as a change 3 4
agent?
Promoting professional Question 9: How confident were you in applying 3 4
development principles of self-awareness?

Question 10: How confident were you in 3 4

encouraging evidence-based
practice?
Question 11: How confident were you in applying 3 4
leadership theory to practice?

We achieved our objective of evaluating the
effectiveness of simulation to increase learner
confidence. The debriefing sessions, faculty
feedback, and peer feedback contributed to the
improvement. This change is viewed as a positive
step in narrowing the gap in evidence that supports
the use of simulation to teach advanced leadership
skills. We learned that graduate students are eager to
engage in opportunities that enhance cognitive
skills, improve verbal communication, and
strengthen collaborative skills.
Patrician and collegues11 studied the
developmental needs of charge nurses and
discovered that nurses are often promoted to
leadership positions without receiving adequate
training. As a result, the researchers recommended
that both graduate and undergraduate programs be
revamped to include simulation and observation
experiences that prepare nurses for high-performing
leadership positions. This QI project aligns with the
recommendations outlined by Patrician and
colleagues.11 Furthermore, the project findings add
to the limited body of evidence suggesting that
simulation is a key component in increasing nurse
leader competency.12
There are several limitations of this QI
project. First, the sample size consisted of 8
participants, and the simulation was
conducted in a single setting, making it
difficult to generalize the findings. Second,
the personal expectations of the MSN in NL
learners, combined with faculty oversight,
were utilized to develop the simulation
scenarios instead of using tested and
validated simulation scenarios. Finally, the
success of the simulation and debriefing
depended on the learner’s attitude, level of
engagement, and the ability to build upon
prior knowledge.
Lessons Learned
Several lessons were learned from this
project. The first was the importance of
having the scenarios reflect realistic
situations that would be faced by FLNLs.
The learners noted that this was one of the
most valuable benefits of the simulation. A
second lesson was the need for debriefing
and providing structure to that debriefing.
Although consistent questions were asked of
all learners, more structure will be added for
future simulation scenarios. A final lesson
learned was the significance of providing
the learners with tools to be successful. By
having the learners read Crucial
Conversations8 prior to the simulation
activity, application of the principles in the
book could be practiced during the
simulation. Further refinement of this
approach will be applied during future
simulation activities.

CONCLUSIONS Keating SB, DeBoor SS, eds. Curriculum Development and

The use of simulation, debriefing, peer
feedback, and faculty feedback led to
improved learner confidence in the
measured AONL Nurse Manager
Competencies.1 Our findings have
implications for both faculty and
nurseexecutivestaskedwitheducatingorhirin
gqualified nurse leaders. Faculty and
hospital nurse leaders should consider using
a dyad approach to increase learner
confidence. Leadership simulations such as
those used in the project are an opportunity
for partnerships between practice settings
and academia. Further work should explore
options for utilizing the education
department staff within health care facilities
to build upon knowledge gained during
graduate and undergraduate nurse
leadership simulation experiences.
Perhaps, this approach
will facilitate the seamless
transition of nurses into
leadershiproles.Morestudiesareneededtoexpl
oretheuseof
simulationingraduateandundergraduateprogr
ams and its effect on nurse leader behavior.
Evaluation. New York, NY: Springer Publishing; 2018:107-
121.
8. Patterson K, Grenny J, McMillan R, Switzler AI. Crucial
Conversations: Tools for Talking When Stakes Are High. 2nd
ed. New York, NY: McGraw Hill; 2011.
9. Najjar RH, Lyman B, Miehl N. Nursing students’ experiences
with high-fidelity simulation. Int J Nurs Educ Scholarsh.
2015;12(1):27-35.
10. Sundler AJ, Pettersson A, Berglund M. Undergraduate
nursing students’ experiences when examining nursing
skills in clinical simulation laboratories with high-fidelity
patient simulators: a phenomenological research study.
Nurse Educ Today. 2015;35(12):1257-1261.
11. Patrician PA, Oliver D, Miltner RS, Dawson M, Ladner KA.
Nurturing charge nurses for future leadership roles. J Nurs
Adm. 2012;42(10):461-466.
12. Waxman KT, Delucas C. Succession planning: using
simulation to develop nurse leaders for the future. Nurse
Lead. 2014;12(5):24-28.
Linda Cole, DNP, RN, APRN, CCNS, CNE, CPHQ, is an
Assistant Professor of Nursing, Clinical at the Cizik School
of Nursing at The University of Texas Health Science
Center in Houston, Texas. She can be reached at
Linda.Cole@uth.tmc.edu. Lisa Boss, PhD, RN, APRN. CNS,
CEN CNE, is Associate Professor, Clinical, Debra Fowler,
PhD, MBA, RN, CNE, NEA-BC, BCC, is Associate Professor,
Clincial, and Latarsha Cheatham, DNP, RN-BC, APRN,
FNP- BC, is Assistant Professor, Clinical at the Cizik School
of Nursing.

Note: This project did not receive any specific grant

REFERENCES from funding agencies in the public, commercial, or
1. American Organization of Nursing Leadership (AONL).
not-for-profit sectors. There are no conflicts of
AONL Nurse Manager Competencies. Chicago, IL: AONL;
2015. interest to declare.
2. Welch TD, Strickland HP, Sartain AF. Transition to nursing
practice: a capstone simulation for the application of 1541-4612/2021/$ See front
matter Copyright 2020 by Elsevier
leadership skills in nursing practice. Teach Learn Nurs.
Inc. All rights reserved.
2019;14(1):283-287.
https://doi.org/10.1016/j.mnl.2020.08.012
3. Lewis R, Strachan A, Smith MM. Is high-fidelity simulation
the most effective method for development of non-
technical skills in nursing? A review of current evidence.
Open Nurs J. 2012;6:82-89.
4. Radovich P, Palaganas J, Kiemeney J, Strother B, Bruneau
B, Hamilton L. Enhancing leadership orientation through
leadership simulation. Best Pract Crit Care. 2011;31(5):58-
63.
5. Sharma RK. Emerging innovative teaching strategies
in nursing. J Nurs Health Care. 2017;1(2):1-3.
6. Bada DR, Olusegun S. Constructivism learning theory: a
paradigm for teaching and learning. IOSSR J Res Method
Educ. 2015;5(6):66-70.
7. Mennenga HA. Implementation of the curriculum. In:
A Nursing Leadership Practicum in the Time
of COVID19:
A Southeastern University Experience
Kate Jones, DNP, RN, CENP, CCM, Liam C. Hein, PhD, RN, FAAN, and
Lisa James, DNP, RN, MHA, NEA-BC

Coronavirus disease 2019 (COVID19) shutdowns have impacted nursing education,

particularly student clinical experiences. In this paper, we discuss how we adapted an
accelerated MSN nursing administration clinical course to meet the needs of our practicing
students and the Commission on Collegiate Nursing Education (CCNE) standards in the
context of the COVID19 pandemic. Resources available through the American Organization
for Nursing Leadership (AONL) were utilized. Students reported a supportive, valuable
learning experience they could apply in their future practice as nurse leaders. Considerations
for future revisions to the course and to the AONL Guiding Principles are also presented.

I
n the short time it has infected humans, the novel
coronavirus identified as COVID-19 has had an
enormous impact on higher education. Colleges of
nursing have faced unique challenges in both prelicensure
and graduate programs. Nursing administration students
need experiential learning in a clinical setting to prepare
them for their professional role and possible advancement.
In response to the pandemic, and to prioritize the health
and safety of staff, patients, and students, many clinical
sites halted all student rotations in spring 2020.
Recognizing the disruption that this was causing
nationwide, the Commission on Collegiate Nursing
Education (CCNE) provided guidance for its accredited
programs, stating: “while CCNE expects that all students
will meet program outcomes, flexibility in clinical
hours and types of experiences, even if not previously
used by the program, are acceptable.”1 The objective of
this work was to create a practicum experience for
students in our master’s degree program in nursing
administration that retained our standards of excellence
while providing the flexibility that
was necessary under the circumstances.
the course to 6 weeks’ duration. The course was already
distance-accessible so didactic content could be delivered
online and asynchronously as planned. The challenge was
how to provide experiential learning for this cohort when
clinical sites were no longer allowing students in their
facilities. Additionally, as practicing nurses, many of the
students themselves were on the frontlines of the
pandemic response. It would not be possible for students
to complete 112 practicum hours and a leadership project
as planned, but the goal of providing a meaningful
learning experience had not changed. Two faculty
members and 1 part-time adjunct were assigned to the
course. With the support of the academic dean, a decision
was made that instead of the normal projectfocused
practicum, students could learn through the

THE PROBLEM
At a college of nursing within a public university in the
Southeastern United States, students pursuing a master’s
degree in nursing with a focus on nursing administration
and leadership were about to begin a 3credit-hour, 7-week
course that included 112 hours of practicum experience
when the university made a decision to suspend all in-
person courses. University decisions further compressed
 KEY POINTS
demic has required significant changes to nursing education. In a crisis situation, graduate education for future nurse leaders can successfully be modifie
g can take place in the midst of
mstances.
lived experience of being a registered nurse and a
graduate student during the early stages of a
worldwide pandemic.
THE SOLUTION
Locating and using resources effectively is an
expectation of nurse leaders, and we often look to
professional organizations to find resources for
professional and personal development. A search
was conducted for resources about how nurse
leaders should respond in a crisis. The American
Organization for Nursing Leadership (AONL)
provides guiding principles to help nurse leaders
manage a variety of issues. One of the available
documents on AONL’s website was the 2017 Role
of the Nurse Leader in Crisis Management.2
Jointly developed with the AONL (formerly
AONE, American Organization for Nurse
Executives) Crisis Management Taskforce and
the American Hospital Association’s Society for
Healthcare Safety and Market Development
(SHSMD) the brief 3-page document provided
an outline upon which to rebuild the course in the
time of COVID. The AONL document is
organized into 5 priority focus areas; these
provided the outline for the active learning
portion of the course. The goal was to have
students observe, assess, analyze, and apply the
principles in their specific setting, and to use
reflection to write about their experiences. Each
week’s assignment was based on the focus area
for the week. It was recognized that students
were living through a stressful time and the
assignments were designed to connect what they
were experiencing to master’s level nurse
leadership. It was also important to maintain
standards of excellence for the course by
incorporating the Essentials of Master’s Education
in Nursing,3 as well as the AONL Nurse Manager
Competencies.4 The writing assignments are
provided in Table 1.
Synchronous Interaction
In prior versions of the course, the student worked
with a preceptor who served as a guide and
facilitator for a leadership project in the clinical
setting. The student and preceptor met at least
weekly. To duplicate this connection and support,
course faculty conducted online synchronous
meetings using Zoom technology. In order to
accommodate student work schedules, 2 options
were offered: Thursdays at 8 p.m. or Saturdays at
6:30 p.m. The meetings were not mandatory
because some students would simply not be able
to join due to work, family, or self-care needs.
The topic for each meeting was the previous
week’s priority focus area.
Caring for and About Students
Caring is one of the college’s core values, and in
the past year, there has been a focus on identifying
faculty behaviors that demonstrate caring in the
online learning environment. A seminar and
toolkit were provided to all faculty during the fall
semester. Caring behaviors were intentionally
used throughout the course. It was important to
acknowledge students’ stressors and to provide
extra support and encouragement during these
very unusual circumstances. At the same time,
faculty demonstrated caring by remaining
committed to an excellent learning experience and
by demonstrating empathy, opportunities for open
dialogue, and building community.
RESULTS Demographics
Thirty students were registered for this class. One student
dropped out during week 2, reporting that the revised
clinical experience did not meet their learning needs; the
remaining 29 students successfully completed the course.
All but 1 of these students were working either full or part
time in nursing at the time. Clinical specialty areas
represented by this group of students included inpatient
settings (emergency departments, operating room, cardiac
care, neonatal care, oncology, and behavioral health) and
outpatient settings (public health, outpatient diagnostic
testing, same-day surgery, hospice, and behavioral health).
Some students were in nursing education, case
management, and supervisory or administrative roles,
whereas others were in staff nurse roles and preparing for
their first leadership opportunity.
The National Council of State Authorization
Reciprocity Agreements (NC-SARA) provides state
oversight of distance education at the post-secondary
level, but does not address licensing board approval for
nursing or other professional programs.6 Some state
boards of nursing or higher education commissions have
placed restrictions on participation in distance nursing
education. Currently, students from 20 states can be
accepted in this graduate program. This cohort included
students from 9 states: Georgia, Maryland, New York,
North Carolina, Pennsylvania, South Carolina, Texas,
Virginia, and Washington. At the time this course was in
session (late March to early May 2020), some of these
geographic areas were more significantly impacted by
COVID-19 than others. This is important to note
because each students’
 experience was different, but the course design needed to
accommodate all circumstances.
Participation
Student participation in the writing assignments had a
100% completion rate. The class week started on Monday
morning, and assignments were due by Sunday night. There
were fewer than 5 instances of late submission. If a
student did not submit an assignment on time, faculty
contacted the student to check in. In some cases, there was
a COVID- related reason that the assignment could not be
completed on time. Ideally,

146 April 2021 www.nurseleader.com

Week Focus Area Assignment
2 Tenets of crisis communication 400- to 500-word essay
Evaluate the COVID-19 communications that have occurred in your
clinical setting. Assess how well the communication, especially from
nursing leaders, has adhered to the tenets of crisis communication
listed. Provide specific examples. Identify 1 communications strategy
that you have observed that you will apply in your leadership practice.
NOTE: If you are not currently practicing, or if your setting has had
minimal impact, select a nursing organization or a county or state
response to write about. The key factor is that nursing leadership/
communication is involved.
3 Nursing leadership behaviors 200- to 250-word essay
Consider the nursing leadership behaviors listed. Discuss the
importance of these behaviors. As an employee, how important are
these to you? During this pandemic, have you seen a nursing leader
display these behaviors?
E-mail
Write an e-mail to a nurse leader who you have seen display these
behaviors, acknowledging and thanking them for their leadership.
Actually sending the e-mail is a personal choice; for the assignment,
submit a copy—but hit “send” only if you want to.

4 Necessary nursing leadership skills 400- to 500-word essay

One of the necessary nursing leadership skills listed in this focus area is
that a nurse leader “approaches a crisis from a systems theory
perspective.”
Read the attached article by Cordon (2013).5 Consider what you have
observed over the past few weeks in this pandemic. Have you seen
nursing leaders apply systems thinking or a systems theory perspective in
their response? Give specific examples of your observations, whether the
answer is yes or no. In your role, have you applied systems thinking
during this crisis? What have you learned about your own ability to be a
“systems
thinker”?
5 Priorities of a crisis readiness plan Analysis
Prior planning is an element of crisis readiness. From your perspective as
an employee in a clinical organization, how effective was the readiness
training and education that you participated in? Part 1: In bullet point
format, list both the strengths and opportunities for improvement. Part
2: Now put your nurse leader hat on. What are 3 action items that you
would implement in the next year to improve readiness for future
events. No specific word count is required for this assignment.
 6 Nurse leader’s role Review the AONL (2017) document and consider the 3 bullet points
listed under the heading “Nurse leader’s role.” Reflection
(150 to 200 words)
Comment on what you have observed during this pandemic from
nursing leadership as relates to these 3 descriptors of the nurse leader’s
role (NOTE: If you are not currently practicing, or if your setting has had
minimal impact, answer this question from a different viewpoint—a
nursing organization, or local/state/federal leader. The key factor is that
the nursing leader’s role is addressed).
Critical thinking (150 to 200 words)
What 1 additional bullet point would you add to this list, and why?

the student would have contacted faculty in
advance, but faculty were understanding and
supportive during this time of heightened stress
and vulnerability.
Participation in Zoom meetings was moderate,
with 6 to 9 students per session. These sessions
started with an opportunity for open dialogue,
providing students somewhere safe to verbalize
what they were seeing and feeling. The
conversation then transitioned into the topic of the
week. Students who did participate expressed
appreciation for the connection and the
opportunity to share.
The course typically requires 112 hours of
practicum experience over 7 weeks. In this term,
the calendar was compressed to 6 weeks, because
the start date was delayed while key decisions
were made at the university level. Faculty faced
the important decision of how to address this
requirement. The program is accredited by CCNE,
which requires planned clinical experiences that
“enable students to integrate new knowledge and
demonstrate attainment of program outcomes.”7
The standard does not specify how many hours are
required. Unlike nurse practitioner preparation
programs, which require 500 direct clinical hours, 8
the number of indirect clinical hours required for
the nursing administration program is at the
discretion of the college. Course faculty decided
to allow students the flexibility to complete as few
or as many hours as they could manage. For an
experience to count as clinical hours for learning
purposes, it had to be separate and apart from their
“normal” work. Many of these students found
themselves expanding their normal roles due to
their organization’s needs during the pandemic.
Recognizing these activities as clinical hours for
learning purposes was appropriate. If students
were involved in the planning, implementation,
and/or evaluation of the response to the pandemic
in their setting, especially interprofessional
efforts, they were allowed to count those hours.
Once something became their “normal” day-to-day
responsibility— even if it was COVID related—it
could not be counted. Extra webinars, trainings,
and meetings could be included. All time devoted
to writing weekly assignments counted, as did
participation in Zoom meetings. In normal practice,
the preceptor and faculty approve clinical logs at
the end of the semester. We asked students to
track their hours and submit logs as usual. All
students submitted logs and reported a mean of
61.31 hours over the 6 weeks (range = 13 to 136
hours, SD = 37.88 hours, median = 42 hours).
Content
Students took the writing assignments (Table 1)
seriously and provided strong responses to the
questions posed, particularly about systems
thinking during a crisis response. This reflects that
they have met the MSN essential stating that
graduates are prepared to “demonstrate the ability
to use complexity science and systems theory in
the design, delivery, and evaluation of health
care.”3(p.12) Of particular interest was the response
to our last question: “What 1 additional bullet
point would you add to this list, and why?” In the
original AONL document, there are 3 bullet points
under the category of “Nurse Leaders Role.” They
are:
Critical contributor to a crisis readiness plan by
leveraging a nurse leader’s broad scope of influence

Collaborator and effective member of the senior complexities of patient care2

leadership team
A theme emerged among the responses, suggesting that a
Trusted patient advocate who understands the bullet point could be added in regard to the nurse leaders’ role
as an advocate for staff and team members. Some
examples of how this was expressed included:
Trusted advocate and voice for clinical nursing staff
Trusted nurse advocate who understands the complexities
of nursing
Be a trusted team member advocate who understands how
a crisis may affect the team member's stability and
health
Student Feedback
Qualitative feedback was obtained using the normal end-
of- course evaluation process. Student participation is
encouraged, but not mandatory, and the responses are
anonymously gathered using the software Class Climate.
The response rate was 82.8% (24 of 29 students). Overall
course evaluation using a scale from 1 (negative) to 4
(positive) was 3.76 (SD = 0.40; college mean = 3.62).
Students were also given an opportunity to provide
comments. One student stated, “This has been an
extremely stressful time in health care, for all of us. I
really valued the ability to have our learning meet us at the
point of crisis, because it required me to stop and think on
purpose. Many times due to stress or exhaustion, we do
not reflect well upon our experiences which can lead to
missing great opportunities” (anonymous survey response,
May 2020).
Faculty also offered students a forum titled
“Farewells” on the class discussion board, with
instructions to use this optional forum to say goodbye to
classmates and include your biggest takeaway from the
course. Seventeen students participated (58.6%). Student
comments included the following:
My hope is that going through this will make us better
nurses and better leaders.
This course has exceeded my expectations, and the
information was invaluable and applicable to my
everyday work. The core of systems thinking is a
valuable concept. I will definitely take this knowledge
with me in my professional journey.
Thank-you all for the encouragement, support, [and]
engaging and thought-provoking content this semester.
We were able to learn and use crisis management skills
firsthand. The takeaway is that when you put your
systems thinking cap on, you can successfully
accomplish anything.
This community really rallied together, and I found it to
be a wonderful course that I was able to learn so much
from.
DISCUSSION
Course faculty were faced with the challenge of
maintaining high standards for learning while adjusting to
the realities of providing graduate nursing education
during a pandemic. The AONL Guiding
Principles for nursing leadership’s role in crisis
management provided an organizing template for a
redesign of the student’s practicum experience that could
be deployed quickly. Throughout the compressed 6-week
course, students were engaged and committed to learning,
and faculty were flexible and caring in their approach.
These elements combined to produce a successful term
despite the disruption of the pandemic.
Looking forward faculty and leadership in nursing
programs need to plan for the potential of an extended
pandemic response, and for changes that will be needed in
a post-COVID world. The course revisions described in
this report happened very quickly. Now that there is an
opportunity to plan ahead, we can apply what worked well
and update the course to include some of the elements that
were successful. For example, students recommended that
optional Zoom meetings should continue. Faculty agreed
that crisis management, which was previously only briefly
addressed in this program, should be included in the
learning objectives. We also suggest that on a broader
scale, AONL could revisit the guiding principles after the
pandemic response has been sufficiently evaluated to
determine whether they served nursing leaders well, or if
there is an opportunity to update and revise them.
 2020. Available at: https://www.aacnnursing.org/Portals/42/
CCNE/News/CCNE-Statement-on-Coronavirus-Bacc-and-
Grad. pdf. Accessed May 26, 2020.
2. American Organization for Nursing Leadership. AONL Guiding
Principles: Role of the Nurse Leader in Crisis Management.
2017. Available at: https://www.aonl.org/system/files/media/
REFERENCES file/2020/03/aonl-role-of-the-nurse-leader-in-
crisismanagement_0.pdf. Accessed May 26, 2020.
1. Commission on Collegiate Nursing Education. Information
Regarding Coronavirus Impact on CCNE-Accredited 3. American Association of Colleges of Nursing. The Essentials of
Baccalaureate and Graduate Nursing Programs. Updated Master’s Education in Nursing. 2011. Available at: https://
March 14,
4. www.aacnnursing.org/Portals/42/Publications/
MastersEssentials11.pdf. Accessed May 26, 2020.
5. American Organization for Nursing Leadership. AONL Nurse
Manager Competencies. 2015. Available at:
https://www.aonl.
org/system/files/media/file/2019/06/nurse-
managercompetencies.pdf. Accessed May 26, 2020.
6. Cordon CP. Systems theories: an overview of various system
theories and its application in healthcare. Am J Systems
Science. 2013;2(1):13-22.
7. National Council of State Authorization of Reciprocity
Agreements About NC-SARA. 2020. Available at:
https://www.ncsara.org/about-nc-sara. Accessed May 26,
2020.
8. Commission on Collegiate Nursing Education. Standards for
Accreditation of Baccalaureate and Graduate Nursing
Programs. 2018. Available at:
https://www.aacnnursing.org/Portals/42/
CCNE/PDF/Standards-Final-2018.pdf. Accessed May 26, 2020.
9. National Task Force on Quality Nurse Practitioner Education.
Criteria for Evaluation of Nurse Practitioner Programs. 5th ed.
A Report of the National Task Force on Quality Nurse
Practitioner Education. 2016. Available at:
https://cdn.ymaws.com/www.
nonpf.org/resource/resmgr/Docs/EvalCriteria2016Final.pdf.
Accessed May 26, 2020.

Kate Jones, DNP, RN, CENP, CCM, is an assistant

professor and program director at the University of
South Carolina College of Nursing in Columbia. She can
be reached at jones99@mailbox.sc.edu. Liam C. Hein,
PhD, RN, FAAN, is an associate professor at the
University of South Carolina College of Nursing and an
Atlantic Fellow for Health Equity. Lisa James, DNP, RN,
MHA, NEA-BC, is the interim associate nurse executive
and interim director of the emergency department at
Prisma Health–Richland in Columbia and adjunct
faculty at the University of South Carolina College of
Nursing.

Note: This research did not receive any specific

grant from funding agencies in the public,
commercial, or not-for-profit sectors.

1541-4612/2021/$ See front

matter Copyright 2020 by
Elsevier Inc. All rights reserved.
https://doi.org/10.1016/j.mnl.2020.06.010
Nurse Education Today 98 (2021) 104748

Fostering leadership competence and satisfaction in nursing undergraduates through a student-

led conference: A quasi-experimental pre-post study
M.A. De Juan Pardo*, P. Fuster, A. Gallart, E. Rodríguez, L. Wennberg, M.L. Martin-Ferreres
Nursing Department, Faculty of Medicine and Health Sciences, Universitat Internacional de Catalunya, Sant Cugat del Vall`es, Spain

ARTICLEINFO ABSTRACT
Keywords: Background: Numerous benefits have been reported for student-led conferences, such
Communication skills as increased leadership. This competence has been recognized as important for nurses
Critical thinking so as to ensure the provision of safe and high- quality care in complex environments.
Emotional intelligence However, research has yet to examine empirically the impact of student-led
Leadership conferences on students’ leadership behaviours.
Nursing students
Student-led conferences Objectives: To examine the impact that participation in a student-led conference had
Teamwork on the self-perceived leadership competence of nursing undergraduates.
Design: Quasi-experimental single group pre-post intervention study.
Setting: Faculty of Medicine and Health Sciences at the Universitat Internacional de
Catalunya.
Participants: 31 students enrolled in two elective modules offered during the final year
(fourth year) of a nursing degree programme.
Methods: Pre-post assessment of self-perceived leadership behaviours among nursing
students involved in planning and organizing a scientific conference. In addition to
carrying out the tasks of organizing the Conference, all students participated as co-
authors of an oral communication, thus being able to develop both cognitive and non-
cognitive domains. Leadership was measured using ES_SALI scale, the Spanish
version of the Self- Assessment Leadership Instrument.
Results: Involvement in the student-led conference led to a statistically significant
increase in self-perceived leadership competence among nursing undergraduates (p <
.001). Both the total ES_SALI score and scores on each of its four dimensions
(Strategic thinking, Emotional intelligence, Impact and influence, and Teamwork
skills) increased significantly, and the percentage change was above 8% in all cases (p
< .01). The greatest increase (10.99%) corresponded to the ‘Impact and influence’
dimension of leadership. Conclusions: The results suggest that student-led
conferences are an effective way of
helping nursing undergraduates to develop their leadership competence.
1. Introduction role (Martinez Estalella et al., 2020; Shingler- Nace, 2020).
Indeed, with increasing numbers of frail and elderly
The complex and ever-changing demands of patients, many with multimorbidity, leadership by nurses is
healthcare environments have been further an important element in the provision of safe and high-
highlighted during the COVID-19 outbreak, quality care (Glazer et al., 2011).
underlining the need for nurses to be capable of The need for nurses to develop leadership capacity has been
showing leadership when carrying out their clinical
recognized by the American Association of Colleges
of Nursing (AACN, 2008), and it is generally agreed
that this process should begin during their university
training (Galuska, 2015; Ward et al., 2016). One of
the methods proposed as a way of promoting their
competence in this respect involves student-led
conferences (SLC) (Galuska, 2015; Ward et al.,
2016), whereby students are required to plan, prepare
and participate in a scientific event. To our
knowledge, however, research has yet to examine
empirically whether this kind of initiative is effective
in helping students develop their leadership
behaviours.
2. Background
Nurse education needs to adequately prepare future
professionals for the challenges they will face when
seeking to provide high-quality patient care in the
context of complex social changes. One aspect that is
considered important in this respect is the
development of their capacity for leadership skills
(Glazer et al., 2011).
Despite the recognition of the importance of
developing leadership competence during nursing
studies, few published studies have been found that
report the results of strategies aimed at developing
leadership (Morrow, 2015). Among these are peer
mentoring (Bright, 2019; Won and Choi, 2017),
leadership courses (Jones and Sackett, 2009),
Dedicated Education Units (Galuska, 2015), and
simulated Psychosocial Role-Playing (Liebrecht and
Montenery, 2016).
Another activity, which can be developed in the
curriculum that may favour the acquisition of
leadership behaviours is attending and participating
in scientific conferences. These events are important
for keeping up to date with the latest developments in
knowledge and evidence-based practice. On the one
hand, attendance at congresses has been shown to
provide the opportunity to listen to experts on a
topic, interact with them and engage in networking,
favouring evidence-based knowledge, as well as the
development of communication skills (Kruse et al.,
2020; Sousa and Clark, 2017). This benefit is
increased in the case of participating as speakers
(with a communication or similar). Authors such as
Asbury (2017), highlight that this allows obtaining
feedback and refinement on the research itself,
and
establishing links with potential collaborators and
leaders in the field. On the other hand, there are
studies that show that participating in the
organization of scientific conferences provides
added benefits, such as the development of
organizational skills (Tomazou and Powell, 2007) or
leadership competence, among others (Rahman et
al., 2015; Tomazou and Powell, 2007). In this
context, the SLC are found to allow students to both
participate and organize scientific conferences
(Leadbeatter and Gao, 2019).
Through the review of the literature, few studies
exist that analyze the impact of SLC on students.
Some publications of SLC experiences have been
found in different countries, within doctoral studies
(Tomazou and Powell, 2007), as well as in the
context of undergraduate studies (Adelekun et al.,
2019; Chua et al., 2015; Leadbeatter and Gao, 2019;
Smith et al., 2013).
Leadbeatter and Gao (2019) detailed the experience
of an SLC organized by senior students in an
undergraduate oral health therapy program in the
Faculty of Dentistry of the University of Sydney
(Australia). The sample consisted of 32 students who
were divided into 3 groups: planning team, media
team, scientific team; in order to organize a single-
day conference, within a subject that had a duration
of 13 weeks. In addition to participating in the
organization, all the students were divided into
groups for the elaboration and presentation of
Abstracts and digital posters. Based on this
experience and the literature reviewed, Leadbeatter and
Gao (2019) proposed a model that includes 12
aspects that serve as a guide for designing and
implementing a SLC successfully.
Accordingly, and consistent with the fact that a core
objective of university training is to foster
professional competences (Gonzalez and ´
Wagenaar, 2005; Llaurado- Serra et al., 2018),
various authors have suggested that SLC be built
into the curriculum as an innovative strategy for
developing students’ communication, networking
and organizational skills (Leadbeatter and Gao,
2019; Rahman et al., 2015; Tomazou and Powell,
2007). It is considered that initiatives of this kind
offer students a unique opportunity to experience
interprofessional learning and to connect to others as
part of a team (Adelekun et al., 2019; Leadbeatter
and Gao, 2019), which as we have already noted are
key dimensions of leadership (Fuster Linares et al.,
2020).
As regards research on the impact of SLC,
depending on the theme in question, it can help
facilitate a change in students’ perceptions and
values. In this sense, Adelekun et al. (2019) reported
that participants in a student-organized conference
on the impact of racism in healthcare went away
with a better understanding of its impact and felt
more connected to other health professionals
who were working to promote anti-racist
conference, where medicine and nursing organized
the 9th Student Medical-Nursing Education
Conference 2013, about Interprofessional Education
(IPE). They found that the attitudes of nursing and
medical students improved towards IPE.
The study by Ward et al. (2016) described a
collaborative student leadership conference and
reported that participants were satisfied with the
leadership competence they had acquired as a result.
However, the authors did not collect specific data on
students’ leadership abilities before and after the
activity. To our knowledge, research has yet to
examine specifically and empirically whether SLC
are an effective way of enhancing students’
leadership competence.
In order to incorporate leadership evaluation into
nursing studies, it is necessary to position oneself
with the leadership style wanted to develop. As
Scully (2015) points out, leadership is a complex and
multidimensional phenomenon. There is currently no
universally accepted definition or theory of
leadership. However, it distinguishes two
predominant styles, transactional leadership, typical
of management, and transformational or connective
leadership, to motivate and inspire change and
mediate towards a shared vision in an empowering
environment (Scully, 2015).
In this sense, a recent article defines ‘shared
leadership’ as the leadership shared among different
members of the interprofessional team, in which the
notion of mutual reciprocity is incorporated (Fuster
Linares et al., 2020). The leader acts as a facilitator,
and this role can be filled by different people, and at
different times (Scully, 2015). Fuster Linares et al.
(2020) identified four dimensions of transformational
leadership that should be developed in nursing
students, namely impact and influence, emotional
intelligence, teamwork, and strategic thinking. Based
on this ‘shared leadership’ model and the leadership
dimensions identified by Fuster Linares et al. (2020)
the SLC was proposed as an activity in the Nursing
Degree curriculum to develop leadership in fourth-
year students at a Spanish University.
3. Purpose of the study
The primary aim of this study was to assess the
impact that participation in a student-led conference
had on the self- perceived leadership competence of
nursing undergraduates. We also evaluated the
degree of satisfaction reported by participants in the
event.
practices. For their part, Chua et al. (2015) described
the experience of a student-led
4. Methods
4.1. Design, sample, and setting
A quasi-experimental design was used, in order to
evaluate the effect of a teaching innovation
intervention on nursing students (Polit and Beck,
2014). In this specific case, a quasi-experimental
single group pre- post intervention study designed to
assess the impact of a SLC hosted in the health
sciences campus of the Universitat Internacional de
Catalunya was carried out.
This university is a private non-profit institution that
offers a nursing degree programme within its Faculty
of Medicine and Health Sciences. Using non-
probability convenience sampling we invited all
final-year (fourth-year) nursing students enrolled in
two elective modules (Interdisciplinary Care of Older
Adults and Old Age and Dependency) across two
academic years (2018–2019 and 2019–2020) to
participate in the study (n = 35). Students who gave
signed consent were included in the study.
Participants that did not respond to the SALI
questionnaire correctly were excluded from the
sample.
4.2. Variables and instruments
In this study, the following variables were analysed:
self- perceived leadership competence, gender, age,
and degree of satisfaction of the participants.
Self-perceived leadership competence, is understood
as the ability to influence the behaviour of other
people in their efforts towards goal setting and goal
achievement (Smola, 2003). The instrument used to
measure this variable was the SALI (Self-
Assessment Leadership Instrument; Smola, 2003,
1988), in its Spanish version, which has been shown
to be a valid and reliable tool for assessing
leadership behaviours (Universitat Internacional de
Catalunya). The Spanish version of the SALI,
ES_SALI, comprises 40 items, each rated on a five-
point Likert-type scale from 0 (“I don’t usually
behave in this manner”) to 4 (“I almost always
behave in this manner”), such that the total score
ranges from 0 to 160. The 40 items are distributed
across four dimensions of leadership, labelled as
follows: Strategic thinking (11 items), Emotional
intelligence (8 items), Impact and influence (7
items), and Teamwork skills (14 items). In the
present study, we considered both the total score on
the ES_SALI and the score on each of these four
dimensions.
Gender and age were collected through two
questions that were included at the beginning of the
ES_SALI questionnaire.
The degree of satisfaction felt by participants
in the student-led conference was measured
using an ad hoc self- administered
questionnaire, not yet validated, comprising
an overall satisfaction rating of between 5 and 50.
The questionnaire also included four open questions
asking about aspects that could be improved, the
timing of the event, and proposals for future
conferences of this kind.
five questions, each rated on a 10- point scale from 1
(not at all satisfied) to 10 (completely satisfied),
thus yielding
would consist of, and the topic it would address was chosen.
Students were then split into two groups (scientific
committee and steering committee) and asked to nominate a
president for each, after which an outline of the conference
programme was drawn up, covering the

4.3. Data collection

The coordinator of the subjects from which the

congress was organized was responsible for
distributing and collecting the ES-SALI
questionnaires, which were self- administered on the
first and last day of said subjects. The satisfaction
questionnaires were delivered to participants of the
congress through inclusion with the accreditations
and folders with the program, so that they could fill
them voluntarily and anonymously, delivering them
to the coordinator when leaving the room.

4.4. Intervention

The intervention was a regional student-led

conference, carried out in Catalonia, which was
planned and organized by nursing students enrolled
in the two aforementioned course modules. These are
elective modules available to final-year nursing
students and they involve 30 h of teaching time.
Eighteen of the 30 h were set aside for organizing the
conference, while the remainder were dedicated to
theory lectures and work in the simulation lab on
topics related to interdisciplinary care of older adults.
Classes (two hours, twice weekly) ran from the first
week of September to the first week of November,
the same time that the intervention lasted. The two
parts of the module were assessed separately.
Concerning the conference, students were assessed
on the basis of a group project and an oral
presentation, combined with continuous assessment
of their contribution throughout and the degree to
which they each met the objectives set. The theory
part of the module was assessed using a multiple-
choice exam and a series of short questions.
The timetable for the modules was structured around
Leadbeatter and Gao’s (2019) twelve tips for a
successful student-led conference in health
professional education. However, we omitted their
Tip 8 (Offer a parallel program in leadership,
negotiation, conflict resolution, and communication)
as these aspects are addressed throughout the nursing
degree programme. The use of these tips enables
students to gradually build their knowledge and skills
regarding the organization of events of this kind.
The first class began with students completing the
ES_SALI. Next, they were told what the conference
following: Welcome and opening address,
round table with invited speakers, oral
communications by students, awarding of the
prize for best communication, summing up
and conclusions from the day (by the
president of the scientific committee) and
closing plenary session. It was also decided
who would moderate the round table and the
oral communications. In addition to being
assigned to either the scientific committee or
steering committee, students were split into
groups of three or four and given the task of
producing an oral communication that would
be presented during the conference.
In the second class, the students and tutor
agreed on a title for the conference and on the
specific topic that would be addressed by
each of the oral communications. The
scientific committee proposed possible
external speakers (for the round table and
plenary sessions) and began drafting the
invitations. Meanwhile, the steering
committee brainstormed possible sponsors
and started work on contacting them, the aim
being to obtain funding that would cover the
costs of the coffee break, gifts for invited
speakers, the prizes for the best
communications, and the conference pack
that would be given to all participants.
Subsequent to this second class and up until
the day of the conference, students had group
tutorials with the tutor in order to monitor
their organization of the event and the
preparation of oral communications. The
aspects, and raising questions that led them to carry
out a critical analysis of their actions. This self-
evaluation was not recorded so as not to condition
the students’ participation, they were only
encouraged to verbalise their feelings and
experiences, in order to help them close the learning
circle (Arthur et al., 2013). At the end of this
feedback, students once again completed the
ES_SALI.
4.5. Data analysis
Quantitative variables from the ES_SALI and the
satisfaction survey were described in terms of means
and standard deviation. Numerical data from paired
groups (pre/post ES_SALI) were compared using
either the paired Student’s t-test or Wilcoxon test,
depending on normality. For the comparison of
numerical variables for unpaired data, we used either
the Student’s t-test or Mann-Whitney U test,
depending on normality. Moreover, these analyses
were stratified by sex and age of students.
Confidence intervals were set at 95% and p values
<.01 were considered significant. All data processing
presidents of the scientific and steering committees
met once a week with the coordinator of the course
module in order to exchange feedback on the
process and to resolve any problems that arose. The
day before the conference, students carried out all
the final preparations: preparing conference packs,
badges and gifts, getting the hall ready, and
rehearsing the various interventions that would take
place during the conference.
The conference itself took place outside the
normal timetable of the nursing degree programme,
outside class hours. The duration of the
Conference was 4 h in the afternoon. Each
student performed the role that he or she had been
assigned with regard to its organization and the
presentation of communications. In turn, some
professors from the Nursing Department
collaborated in the attention to the speakers and were
part of the evaluation panel for the best
communication. At the end of the conference, all
attendees were asked to complete the satisfaction
survey. On the day that students sat for the final
exam for the module, the coordinator also
organized a feedback session in order to transform
the lived action into learning. For this, the teacher
had a planned script aimed at promoting reflective
thinking, applying Kolb’s learning cycle (Kolb,
2015). This process makes it possible to deduce
concepts and principles that derive meaning for the
lived experience and directing behaviour in new
situations. The students were encouraged to carry
out a self-evaluation, exploring in their lived
emotions, both the positive and the negative
and analysis were performed using SPSS v26.
For qualitative variables, namely the responses to the open
questions on the satisfaction survey, we performed a content
analysis using Microsoft Excel 2016.
4.6. Ethical considerations
The study was approved by the Research Ethics Committee
of the university where the research was carried out (INF-
2017-03). All students were given written information about
the study objectives and what their participation would
involve, and no data were collected until they had signed
informed consent. It was made clear to them that
participation was voluntary. In order to preserve their
anonymity, all questionnaires were coded and only the
principal investigator had access to the full data set.
5. Results
Although all 35 nursing students who were enrolled in the
two- course modules were involved in planning and
organizing the conference and took part in it, only 31 (14
male, 17 female; 88.6%) agreed to complete the leadership
instrument (the ES_SALI) at the two-time points. In 2018,
 17 students (7 male and 10 female) completed the students involved in organizing the event, 11
ES_SALI, while in 2019, 14 (7 male and 7 female) students from other courses offered within the health
did so. The mean age of the 31 students was 25.6 sciences campus, 17 lecturers, 15 invited speakers,
years (SD 6.2), with a range from 21 to 49. None of and 8 health professionals. As regards self-
the students stated having previous experience in perceived leadership competence before and after
attending congresses. involvement in the student-led conference, the
The topic chosen for the 2018 conference was results showed a significant (p < .01) increase in
‘Communicating with terminally ill patients’, while in both the total ES_SALI score and in scores on
2019 it was ‘Maltreatment of older adults’. Across each of its four dimensions (Strategic thinking,
the two conferences, there were a total of 89 Emotional intelligence, Impact and influence, and
attendees: the 35 Teamwork skills). The percentage change (after
value − before value / before value) * 100 (Pope,
2008), was above 8% in each case (Table 1),
with the greatest increase
(10.99%) corresponding to the ‘Impact and
influence’ dimension of leadership.
At the pre-test, there were no significant differences
in self- perceived leadership competence by gender
(Table 2) or age (comparing students aged 25 or
over with their younger peers; Table 3). Nor did we
find significant differences by gender or age in the
magnitude of the increase in self- perceived
leadership competence following involvement in the
student-led conference (Tables 2 and 3,
respectively). During the final group tutorial and
feedback session (at the end of the course module),
the comments made by students who had been
involved in planning and organizing the
conference were very positive. They considered
that the experience had helped them to develop
their skills with regard to organization and task
management, teamwork, communication, and
research, and also that it had been an opportunity to
overcome their fear of public speaking and to work
on more personal qualities such as perseverance,
commitment, and patience. Some students explicitly
stated that the process had helped them to develop
their leadership competence. Others considered that
it had been an opportunity for growth not only on a
professional level but also on a personal level. There
was also an appreciation of the tutor’s input
throughout, which they felt had been crucial in
enabling them to successfully navigate what for all
of them had been a novel experience.
Of the 89 attendees across the two SLC, 61
(68.54%) completed the satisfaction survey. The
overall mean satisfaction rating was 9.33 out of a
maximum of 10 (Table 4), and most of the
comments made in response to the open questions
were highly positive. Notably, mention was made
Table 1
Scores for self-perceived leadership behaviours (ES_SALI) before and after
the student-led conference.
Mean (SD) total score on p-Value for pre-post Percentage
the ES_SALI difference change (%)
(n = 31)
Total 121.31 (18.278) <.001b 9.42
ES_SALI:
Prec
 Total 132.74 (13.609)
ES_SALI:
since there are studies carried out in various countries, in
Post which they also have nursing
Table 2
Mean (SD) score on p-Value for pre- Scores for self-perceived leadership behaviours (ES_SALI) by gender.
Percentage Males (n Females Males vs. Males vs. females
each of the four post difference for change (%) for
each dimension = 14) (n = 17) females at pre- pre-post difference
dimensions of the each dimension test p-Value p-Value
Mean (SD) Mean (SD)
ES_SALI (n = 31)

Strategic 32.84 (5.113) .002a 9.71

thinking: Pre Total cES_SALI: 125.57 117.82 .173b .827b
Pre (17.96) (18.31)
Strategic 36.03 (4.151)
thinking: Total ES_SALI: 136.57 129.58
Post (13.88) (12.93) 32.58
Post
Emotional Strategic thinking: 33.14 (5.29)
25.65 (3.592)
intelligence: Pre (5.06) .768a .664a
Prec .003 b Strategic
Post thinking: 36.78
(4.22) 35.41 25.53
(4.11)
8.93
Emotional 27.94 (2.620) 25.78
Emotional (3.02) .739b .630b
intelligence: (4.29)
intelligence:
Post
Prec
Impact and 19.65 (4.637)
influence: Emotional 28.21 27.70
Pre a intelligence:
Post (2.94) (2.39)
.002 10.99
Impact and 21.81 (3.146)
Impact and 21.07 18.47
influence: influence: Pre (3.95) (4.93)
Post .114a .677a
Teamwork 43.19 (7.382) Impact and 22.92 20.88
skills: Pre .006a influence:skills:
Post (2.61)
45.57 (3.31)
41.23
8.75 Teamwork
Teamwork 46.97 (5.913) Pre (6.32) (7.79)
skills: Post .098a .619a
Teamwork skills: 48.64 45.58 (5.72)

Percentage change (after value − before value / before value) * 100 (Pope, 2008. Post (5.90)
How to Calculate Percentage Change. Quick Tip Ser. Ext. Fac. Conduct. Eval. a
t-Test. b
2007–2008).
a Mann-
Paired t-
Whitney.
test. c
Median [1st quartile 3rd quartile] for: Total ES_SALI Pre in males: 128.5
b
Wilcoxon. Bold indicates p < .01. ES_SALI (Universitat Internacional
de Catalunya), comprising 40 items scored on a Likert-type scale from 0 (“I don’t
usually behave in this way”) to 4 (“I almost always behave in this way”), such that [113.25; 141.75]; Total ES_SALI Pre in females: 123.0 [109.0; 133.0]; Emotional
the total score ranges from 0 to 160. Possible scores on each of the four dimensions intelligence Pre in males: 27.0 [21.5; 29.0]; Emotional intelligence Pre in females:
of the ES_SALI are as follows: Strategic thinking, 0–44; Emotional intelligence, 0– 26.0 [24.0; 28.0].
32; Impact and influence, 0–28; Teamwork skills, 0–56.
c
Median [1st quartile; 3rd quartile] for: Total ES_SALI Pre: 128 [110;
135]; Emotional intelligence Pre: 27 [23; 29]. Table 3
Scores for self-perceived leadership behaviours (ES_SALI) by age.
<25 years ≥25 years <25 years vs. <25 years vs.
of the students’ level of learning and the professional (n 17) 25 years at ≥25 years Pre-
organization of the conference: “Thank you! I thought this pre-test p- post difference p-
Value Value
was a highly professional event”. Regarding possible
improvements, it was suggested that the conference Total ES_SALI: Pre 120.12 122.78 (19.27) .6953a .2036b
should be more widely publicized so as to increase (17.93)
attendance by students from across the health sciences Total ES_SALI: 129.06 137.21
Post (13.56) 32.64 (12.72) 33.07
campus. Opinions differed as to the timing of the Strategic thinking: (5.24) (5.13)
conference. Some respondents felt that hosting the Pre .8222a .2614a
Strategic thinking: Post 34.88 37.43
conference during normal timetable hours would Emotional intelligence: (4.15) (3.84)
encourage more students to attend, whereas others Pre 25.23 26.14
(3.58) (3.67) .4946a .8884b
suggested keeping it to a Friday afternoon or even moving Emotional intelligence: 27.59 28.36
it to a weekend. Post (2.74) (2.50)
Impact and influence:
Pre 20.0 (4.40) 19.21
6. Discussion (5.04) .6514a .0826a
Impact and influence: 21.11 22.64
Post (3.12) (3.08)
This study sought to examine the impact that participation Teamwork skills: 42.23 44.36
in a SLC had on the self-perceived leadership competence Pre (7.35) (7.52) .4364a .6619a
of nursing undergraduates. The analysis of data revealed Teamwork skills: Post 45.47
(5.75)
48.78 (5.78)

that the SLC conference is beneficial for the growth of a b

t-Test.
leadership capacity, both at a general level and in each of Mann-
its dimensions. Whitney.

The participants of this study are in an age range between

21 and 49 years. This situation is not an isolated event,
students over 40 years of age in their samples, such as those carried out in New Zealand (Montayre et al., 2019), or in
 Peru (Diaz-Godino et al., ˜ 2019), in which the sample
was made up of 10% and 1.5% respectively of students of
this age. This fact could be explained due to the fact that
part of the students who enter the Nursing Degree have
extensive previous professional experience, most of them
as nursing assistants.
Table 4
Satisfaction ratings (max. score 10) by attendees at the two student-led conferences.
N = 61 attendees
Overall satisfaction with the conference
Organization of the conference
Duration of the conference
Input from and interaction with the conference organizers
Extent to which the conference content was of interest
The level of leadership of nursing students in Spain was
unknown until now, as there was no validated instrument
available that would allow measuring the level of
acquisition of this competence. It is for this reason that
part of the research team validated the ES_SALI scale that
has just been published recently and that has been used to
carry out this study (Universitat Internacional de
Catalunya). This fact motivates that the only possible
comparison of both the basal level of leadership and its
increase must be with nursing students from other
countries and not with students from our own country.
Therefore, we do not know if nursing students in general
in Spain have good leadership skills.
Leadership by nurses is an important element in the
provision of safe and high-quality care in a complex
healthcare environment, and it is therefore important that
their competence in this respect begins to be developed
during their undergraduate training (Galuska, 2015). One
of the pedagogical approaches proposed as a way of
developing competences of this kind is SLC.
In their recent discussion of what makes for a successful
student-led conference in health professional education,
Leadbeatter and Gao (2019) argue that such conferences
are best incorporated around the middle or towards the
end of a study programme because the students involved
will, in order to benefit from the experience, need to have
already acquired a sufficient body of relevant knowledge
and to have begun to develop their interpersonal and
professional skills, such as understanding how groups
function, time management, and leadership. The positive
results obtained in the present study, in which final-year
nursing students perceived an improvement in their
leadership competence following participation in the
student-led conference, support this claim since the
knowledge and skills they had already acquired during the
first three years of the degree programme allowed them to
show considerable initiative and autonomy when
organizing the event. Had the student-led conference been
proposed at an earlier point in the degree programme, it
In this study, no significant differences were found in
would have been necessary to have created a specific
course module with much more input from the tutor in
order to ensure that students had the minimum experience
and skills required. However, it would not have been
possible for them to have reached the level of learning
they had achieved by the final year of their studies, and it
is unlikely that they would have been capable of
Mean (SD) organizing an event of the quality produced here. That
9.33 (0.77) said, it should be noted that even when students are able
9.43 (0.86) to work independently when organizing a student-led
8.85 (1.21)
9.59 (0.94) conference (as was the case of our final-year nursing
9.59 (0.69) undergraduates), tutors still have a key role to play in
offering supervision and helping students resolve any
difficulties they encounter during the process (Leadbeatter
and Gao, 2019; Ward et al., 2016).
Although previous studies have gathered the perceptions
of students and professionals who participated in SLC,
and have concluded as a result that the experience can
help to develop students’ leadership competence
(Leadbeatter and Gao, 2019), we are unaware of any
study that has specifically and empirically examined
whether involvement in a student-led conference leads to
improved leadership behaviours. Therefore, in order to
weigh up the relevance of our findings, we will compare
our results with those of studies that have also used the
SALI to assess leadership in the health education context.
Larin et al. (2011) measured, among other aspects, the
change in leadership abilities of students enrolled in
nursing and physical therapy programmes at McMaster
University (Ontario) and in a physical therapy programme
at Ithaca College (New York). Students were assessed at
the beginning of their professional programme and again
after completing their first full- time clinical placement.
For all three programmes, total scores on the SALI were
higher in the post-clinical assessment, with the specific
percentage increase being as follows: 1.94% for
McMaster nursing students, 1.26% for McMaster physical
therapy students, and 1.64% for Ithaca physical therapy
students. In a study conducted in South Korea, Oh et al.
(2011) assessed leadership (among other aspects) in
nursing students across different years of their academic
programme. For students on a four-year baccalaureate
programme (the closest equivalent to the programme
being followed by nurses in our study), the authors
observed a 4.72% increase in the SALI total score
between years three and four of the programme (mean
scores of 148.3 and 155.3 in years 3 and 4, respectively).
Although these percentage increases are greater than those
reported by Larin et al. (2011), they remain considerably
lower than the 9.42% percentage change in SALI total
score that our nursing undergraduates showed following
participation in a student-led conference. Our results,
therefore, suggest that SLC should be considered as one
of the most effective ways of fostering leadership
competence among nursing undergraduates.
leadership levels, regarding gender and age, neither before nor
after the intervention (Tables 2 and 3). There is little
literature that refers to these variables in relation to
leadership competence in nursing students. However,
contradictory results have been found regarding the
gender variable. On the one hand, the Carlsson (2020)
study identified a significant difference in the self-
reported competence between female and male students in
favour of men. However, the results of the study carried
out by Sonmez et al. (2019) ¨ show that, despite not
finding a significant difference in terms of the level of
leadership in general concerning gender, some of its
dimensions (congruence and common purpose) were
greater in women than in men. Due to these contradictory
results and the scarcity of literature that have studied this
phenomenon, it can be considered that the data are
inconclusive, and therefore more studies are needed.
As in previous reports of SLC (Adelekun et al., 2019;
Rahman et al., 2015; Ward et al., 2016), both our students
and the other participants expressed high levels of
satisfaction with the experience. The specific benefits that
students mentioned in their feedback (i.e. that organizing
and participating in the conference had given them the
opportunity to develop their skills with regard to
communication, task management, teamwork, and
leadership, among other aspects) are also consistent with
the outcomes of similar initiatives in different countries
(Chua et al., 2015; Leadbeatter and Gao, 2019; Rahman et
al., 2015; Tomazou and Powell, 2007; Yee et al., 2016).
This suggests that our results are not limited to the setting
in which the study was carried out, and hence they add to
the international body of knowledge on this topic.
6.1. Limitations
Like all studies, this one is not exempt from limitations,
one of them being the lack of a control group or
comparison group.
Another limitation concerns the relatively small sample
size and the fact that we only included final-year nursing
students enrolled in two specific elective modules. Thus,
although we observed a statistically significant increase in
self-perceived leadership competence following
participation in the student-led conference, further studies
involving a larger sample and a wider range of students
are necessary to confirm this effect. The inclusion of
students from across the nursing degree programme is
also important for ensuring that all undergraduates have
the opportunity to participate in and benefit from student-
led initiatives of this kind. In this respect, we agree with
those authors (Chua et al., 2015; Leadbeatter and Gao, 2019)
who have argued that student-led initiatives should be
built into
Acknowledgments
This study was awarded with a Beca Fundacio Puig in
Teaching ´ Innovation in 2018. The authors acknowledge
the core curriculum, rather than being restricted to
elective modules.
A further issue to consider is that the conferences were
only attended
by a small number of students from other courses offered
within the health sciences campus. Had a wider range of
health disciplines been represented, this may have further
enriched the debates which took place during the
conference and exposed nursing students to a wider range
of questions from different perspectives. A higher number
of attendees would also have implied a greater test of their
organizational skills. Given the possibility that our
students’ leadership competence would have been further
boosted by having to meet these challenges, it seems
important to ensure that future SLC are publicized widely
within the health campus and are held at a time that allows
for maximum participation by other disciplines.
7. Conclusions
SLC provide students with the opportunity to develop
both professional and interpersonal skills, and they appear
to be an effective way of fostering leadership competence
among nursing undergraduates. Although our final-year
students showed considerable initiative and autonomy
when organizing the event, tutors still have an important
role to play in processes of this kind, offering guidance
throughout, as needed, and helping students to resolve any
problems they encounter.
Author statement
MAJP, PF, AG, ER, LW and MLMF all contributed to the
conception and design of the study. Data were collected
by the first author (MAJP). All the co-authors contributed
to the analysis and interpretation of data and to the
preparation of the manuscript, which they all approve in
its present form.
The contents are our original work and have not been
published, in whole or in part, prior to the present
submission. Neither is the manuscript under review
elsewhere. The study was approved by the ethics
committee of the International University of Catalonia
where the research was conducted.
Ethical aproval
The study was approved by the Research Ethics
Committee of the Universitat Internacional de Catalunya
where the research was carried out. Reference number:
INF-2017-03.
Declaration of competing interest
The authors declare no conflict of interest.
the Research Group Grup de Recerca Emergent en Educacio en
Salut UICBarcelona. 2017 SGR ´ 141. This research was also
supported by chair DECIDE Boehringer- UIC.
 The authors are grateful to Queralt Miro for acting as
statistical advisor to this study. We would also like to
thank Alan Nance for translating and editing the final
version of this manuscript.
References
AACN, 2008. The Essentials of Baccalaureate Education for Professional Nursing Practice. Adelekun,
A.A., Beltran, S., Carney, J., Lett, L.A., Orji, W.U., Rider-Longmaid, E., ´ Stokes, D.C.,
Teeple, S., Aysola, J., 2019. Recognizing racism in medicine: a student- organized and
community-engaged health professional conference. Heal. Equity 3,
395–402. https://doi.org/10.1089/heq.2019.0015.
Arthur, C., Levett-Jones, T., Kable, A., 2013. Quality indicators for the design and
implementation of simulation experiences: a Delphi study. Nurse Educ. Today
33, 1357–1361. https://doi.org/10.1016/j.nedt.2012.07.012.
Asbury, E.A., 2017. The importance of conference attendance in developing research
collaborations. Whitireia Nurs. Heal. J. 59–62.
Bright, A.L., 2019. Practicing leadership skills through peer mentoring and teaching: the
lived experience of BSN students. Int. J. Nurs. Educ. Scholarsh. 16, 1–9. https://doi.
org/10.1515/ijnes-2019-0022.
Carlsson, M., 2020. Self-reported competence in female and male nursing students in the
light of theories of hegemonic masculinity and femininity. J. Adv. Nurs. 76, 191–
198. https://doi.org/10.1111/jan.14220.
Chua, A.Z., Lo, D.Y., Ho, W.H., Koh, Y.Q., Lim, D.S., Tam, J.K., Liaw, S.Y., Koh,
G.C., 2015. The effectiveness of a shared conference experience in improving
undergraduate medical and nursing students’ attitudes towards inter-professional
education in an Asian country: a before and after study. BMC Med. Educ. 15, 1–7.
https://doi.org/10.1186/s12909-015-0509-9.
Diaz-Godino, J., Fern˜ andez-Henriquez, L., Pe´ na-Pastor, F., Alfaro-Flores, P.,
Manrique- ˜ Borjas, G., Mayta-Tovalino, F., 2019. Lifestyles, depression, anxiety,
and stress as risk factors in nursing apprentices: a logistic regression analysis of
1193 students in Lima, Peru. J. Environ. Public Health 2019.
https://doi.org/10.1155/2019/ 7395784.
Fuster Linares, P., Rodriguez Higueras, E., Martin-Ferreres, M.L., Wennberg Capellades,
L., Gallart Fernandez-Puebla, A., 2020. Dimensions of leadership in ´ undergraduate
nursing students. Validation of a tool. Nurse Educ. Today. https://
doi.org/10.1016/j.nedt.2020.104576 in press.
Galuska, L.A., 2015. Dedicated education units: partnerships for building
leadership competency. J. Nurs. Educ. 54, 385–388.
https://doi.org/10.3928/01484834- 20150617-05.
Glazer, G., Ives Erickson, J., Mylott, L., Mulready-Shick, J., Banister, G., 2011.
Partnering and leadership: core requirements for developing a dedicated education
unit. J. Nurs. Adm. 41, 401–406. https://doi.org/10.1097/NNA.0b013e31822edd79.
Gonzalez, J., Wagenaar, R., 2005. Tuning Educational Structures in Europe: A Pilot ´
Project by and for the European Commission in the Framework of the Socrates
Programme 1–7. https://doi.org/10.1163/157007490X00133.
Jones, J.M., Sackett, K., 2009. Integrating leadership and management content across the
curriculum: a 3-course approach. Nurse Educ. 34, 204–208. https://doi.org/
10.1097/NNE.0b013e3181b4c4c6.
Kolb, D.A., 2015. Experiential Learning: Experience as the Source of Learning and
Development, 2nd editio. Pearson Education, Inc., New Jersey.
Kruse, J.A., Didion, J., Perzynski, K., 2020. Strengthening student nurses’ sense of
belonging through attendance at a professional nursing conference. AORN J. 112,
227–236. https://doi.org/10.1002/aorn.13141.
Larin, H.M., Benson, G., Martin, L., Wessel, J., Williams, R., Ploeg, J., 2011. Examining
change in emotional-social intelligence, caring, and leadership in health professions
students. J. Allied Health 40, 96–102.
Leadbeatter, D., Gao, J., 2019. Twelve tips for a successful student-led conference in
health professional education. Med. Teach. 41, 750–755.
https://doi.org/10.1080/ 0142159X.2018.1481500.
Liebrecht, C., Montenery, S., 2016. Use of simulated psychosocial role-playing to
enhance nursing students’ development of soft skills. Creat. Nurs. 22, 171–175.
https://doi.org/10.1891/1078-4535.22.3.171.
Llaurado-Serra, M., Rodríguez, E., Gallart, A., Fuster, P., Monforte-Royo, C., De Juan,
M., 2018. Assessing the competences associated with a nursing bachelor thesis by
means of rubrics. Nurse Educ. Today 66, 103–109. https://doi.org/10.1016/j.
nedt.2018.04.009.
Martinez Estalella, G., Zabalegui, A., Sevilla Guerra, S., 2020. Gestion Y Liderazgo De
Los ´ Servicios De Enfermería En El Plan De Emergencia De La Pandemia Covid-
19:
La Experiencia Del Hospital Clínic De Barcelona. Enfermería Clínica.
https://doi.org/ 10.1016/j.enfcli.2020.05.002.
Montayre, J., Dimalapang, E., Sparks, T., Neville, S., 2019. New Zealand nursing
students’ perceptions of biosciences: a cross-sectional survey of relevance to
practice, teaching delivery, self-competence and challenges. Nurse Educ. Today 79,
48–53. https://doi.org/10.1016/j.nedt.2019.05.013.
Morrow, K.J., 2015. Leadership curricula in nursing education: a critical literature
review and gap analysis. J. Nurs. Educ. 54, 367–371. https://doi.org/10.3928/
01484834- 20150617-02.
Oh, K., Ahn, Y.H., Lee, H.-Y., Lee, S.-J., Kim, I.-J., Choi, K.-S., Ko, M.-S., 2011. A study
on Korean nursing students’ educational outcomes. J. Educ. Eval. Health Prof. 8, 3.
https://doi.org/10.3352/jeehp.2011.8.3.
Polit, D.F., Beck, C.T., 2014. Essentials of Nursing Research: Appraising Evidence for
Nursing Practice, 8th ed. Lippincott, Williams and Wilkins, Philadelphia.
Pope, P., 2008. How to calculate percentage change. In: Quick Tip Ser. Ext. Fac. Conduct.
Eval. 2007–2008.
Rahman, F., Farmer, R., Das, S., Vayani, F., Hassan, M., 2015. Highlights of the 1st
Student Symposium of the ISCB RSG UK. F1000Research 4, 10–13. https://doi.org/
10.12688/f1000research.6616.1.
Scully, N.J., 2015. Leadership in nursing: the importance of recognising inherent
values and attributes to secure a positive future for the profession. Collegian 22,
439–444. https://doi.org/10.1016/j.colegn.2014.09.004.
Shingler-Nace, A., 2020. COVID-19: when leadership calls. Nurse Lead. (Apr 29) https://
doi.org/10.1016/j.solener.2019.02.027.
Smith, S., Alexander, A., Dubb, S., Murphy, K., Laycock, J., 2013. Opening doors
and minds: a path for widening access. Clin. Teach. 10, 124–128. https://doi.org/
10.1111/j.1743-498X.2012.00616.x.
Smola, B., 1988. Refinement and validation of a tool measuring leadership characteristcs
of a baccalaureate students. In: Strickland, L., Waltz, C. (Eds.), Measurement of
Nursing Outcomes. Spring Publishing Company, New York, NY, pp. 314–336.
Smola, B., 2003. Self-assessment leadership instrument. In: Waltz, C., LS, J. (Eds.),
Measurement of Nursing Outcomes. Spring Publishing Company, New, pp. 210–
215.
˙ ¨
Sonmez, B., ¨Ispir, O., Azizo glu, F., Hapçıo˘ glu, S.B.,
Yıldırım, A., 2019. Socially ˘ responsible leadership: a study with nursing and
medical students in Turkey. Nurse Educ. Pract. 36, 144–150.
https://doi.org/10.1016/j.nepr.2019.03.014.
Sousa, B.J., Clark, A.M., 2017. Getting the most out of academic conference
attendance: five key strategies. Int. J. Qual. Methods 16, 1–2.
https://doi.org/10.1177/ 1609406917740441.
Tomazou, E.M., Powell, G.T., 2007. Look who’s taling too: graduates developing skills
through communication. Nat. Publ. Gr. 8, 724–726.
Ward, S.L., LaFramboise, L.M., Cosimano, A.J., 2016. Collaborative student
leadership conference. J. Prof. Nurs. 32, S63–S67. https://doi.org/10.1016/j.
profnurs.2016.01.013.
Won, M.R., Choi, Y.J., 2017. Undergraduate nursing student mentors’ experiences of peer
mentoring in Korea: a qualitative analysis. Nurse Educ. Today 51, 8–14. https://
doi.org/10.1016/j.nedt.2016.12.023.
Yee, C., Wu, A.M., Wu, C.M., Thakore, R.V., Greenberg, P.B., 2016. Student-driven
approaches to undergraduate medical research: a peer-led symposium. R. I. Med. J.
(2013) 99, 31–33.
 A

ssroom “Leadership in Nursing” course on nursing students’ achievement and experiences: A quasi-expe
Süheyla Abaan b a Cankiri Karatekin University, Faculty of Health Sciences, Nursing Department, 18200 Cankiri, Turkey b Hacettepe University, Faculty of N

RTICLEINFO ABSTRACT
Keywords: Background: Providing the undergraduate “Leadership in Nursing” course using an innovative approach is regarded
Nursing to be very important in terms of enhancing leadership skills.
Leadership Purpose: The purpose of this study was to evaluate the effect of using the “flipped classroom” strategy on nursing
Flipped classroom
students’ learning achievements and to describe their experiences of this technique.
Traditional teaching
Learning outcomes Method: The course of “Leadership in Nursing” was delivered in flipped classroom format for 20 students while
another 19 were exposed to traditional teaching methods. Therefore, the study was completed with 39 students.
Data was collected using “Student Introductory Information Form”, “Exams (a midterm exam, a final exam and 4
assignments)”, “In-Class Observation Form” and “Student Feedback Form”.
Findings: Students in the flipped classroom group had obtained significantly higher scores than the students in the
traditional teaching group regarding the assignments, the final exam, and overall grade. Concerning the flipped
classroom, students reported flexibility as the most positive aspect and problems related to the system infrastructure
as the most negative aspect.
Conclusion: The usage of a flipped classroom for nursing students in the leadership course provided valuable results
in terms of the students’ exam scores and achieving course objectives. The flipped classroom strategy for the
“Leadership in Nursing” course was shown to be effective in improving abilities such as distinguishing leadership
skills, visioning, developing a new perspective, willingness to share experiences regarding leadership, also enabling
students to work in harmony in group activities and to participate in classroom discussions, which are all
essential elements of
leadership.

Introduction improve leadership skills. However, undergraduate nurses are not

It is very important for nurses to develop their leadership abilities to
provide effective care and ensure patient safety while performing their
daily leadership roles (Al-Dossary, 2017). National Academy of Medicine
(NAM), identified leadership as one of the five basic values that guide
nursing activities (NAM, 2010). In 2010, NAM released a report on “The
Future of Nursing” which expressed that the nurses should be prepared
to meet the needs of different patients while working as a leader, and
providing safe and quality health care both to patients and healthcare
professionals.
Therefore, leadership skills need to be developed for nurses.
Development of these skills can be achieved through formal education
and training programs (NAM, 2010). Curtis et al. (2011) stated that
leadership skills can be developed at the earliest stage of nursing
education.
Furthermore, Cummings et al. (2010) asserted that leadership, modeling
and the implementation of certain training activities and strategies can
sufficiently prepared to be leaders and they lack the necessary
development of leadership skills in today’s complex and ever-changing
clinical environment (Demeh & Rosengren, 2015). Kling (2010) stated
that nursing students do not have leadership skills due to limited
resources and opportunities which are required for different leadership
skills; they ‘follow’ and ‘observe’ instead of ‘leading’ or ‘doing’. We
understand from the results that there is a gap between training and
leadership skills.
In addition, it can be said that conducting nursing education with the
traditional teaching (TT) method also has the capability to hinder the
development of certain leadership skills. The TT method is a teaching
method which is a one-way type of teaching that it’s content is
originated from the instructor and then conveyed to the
student; a method where there is a pressure in terms of
teaching the course curriculum in a given time period and
likewise, the students are expected to take the relevant
course in the exact time and space (Mortensen & Nicholson,
2015). This situation may cause the students to graduate
without developing certain critical thinking skills - which
results in the inability to use the necessary leadership
skills in the work environment - and puts them into a state
which inhibits them to cooperate with their team members
and/or colleagues (Missildine et al., 2013; Towle & Breda,
2014). Sherbino et al. (2013) argue that the TT method is
not very effective for the student in order to gain sufficient
knowledge and skills related to the occupation which they
are being trained for. In the TT method, the educator also
teaches a general and appropriate course with a pre-
determined single level of difficulty; individuals with
higher or lower cognitive capacity than this level cannot
benefit from the education process sufficiently (Betihavas
et al., 2016).
Thus, nursing programs should encourage their students
and integrate leadership learning experiences into the
undergraduate nursing education (Symenuk & Godberson,
2018) and employ innovative strategies which meet
students’ needs (Demeh & Rosengren, 2015). Innovative
strategies to deliver the “Leadership in Nursing” course
can increase the knowledge of nursing students on how to
be a leader and allow them to develop leadership skills
through active learning strategies. The World Health
Organization (WHO), Nurse Educator Core Competencies
report suggested that nurse educators needed to facilitate
active learning, ensure learning outcomes and select and
use appropriate information technologies and materials
(WHO, 2016).
The flipped classroom (FC) approach is one such strategy
that can appeal to leadership in nursing by changing
traditional teaching (TT) from an educator-focused
phenomenon to a student-centered one where students are
responsible for self-directed learning of course information
(El-Banna et al., 2017; Hessler, 2016; Hanson, 2016;
Yacout & Shosha, 2016).
Literature
The constructivist approach is a cognitive learning theory
that enables students to actively build their own learning
processes, based on their previous learning, experiences and
interactions with their environment (Hawks, 2014;
Richardson, 2003). It is an approach where the student
does not take the information directly, but synthesizes and
organizes his/her new knowledge and experiences upon
the pre-existing knowledge framework. The aim is to give
experimental group increased significantly when compared
students the opportunity to learn directly with learning
materials in line with their own desires (Gilboy et al.,
2015). Therefore, the constructivist approach is based on
the active learning theory and is achieved solely through
active learning strategies (Gilboy et al., 2015). With the
constructivist approach, courses are conducted in a more
flexible manner, focusing on experiential learning and
problem-solving while encouraging students to be more
critical thinkers and reach higher levels of the Bloom
taxonomy (Gilboy et al., 2015).
Based on this approach, FC is described as a method in
which students learn theoretical knowledge online at home
and at their own pace in an entirely self-regulated fashion,
while in-class time is devoted to high-level cognitive
activities using active learning techniques (Hayırsever &
Orhan, 2018). FC combines online (out-of-class) and face-
to- face (in-class) techniques and it provides a student
centered, active learning methodology where the educator
also guides the students instead of only lecturing them,
while the students realize their own learning experiences
and find answers to their questions (Gillispie, 2016;
Hessler, 2016). The use of FC in nursing education fosters
professional competence by preparing nursing students for
critical thinking and effective communication, for the
appliance of therapeutic nursing interventions. Moreover,
research and management, collaborative learning and self-
directed learning (Green & Schlairet, 2017), has the
potential to solve the education-clinical practice divide
(Anolak et al., 2018). Besides these, FC helps students to
synthesize, analyze, apply and evaluate information
usually via combining online and in-class activities
(Maxwell & Wright, 2016). These professional
competences are compatible with the “Leadership in
Nursing” course because students must develop leadership
skills.
In the literature, a quasi-experimental study which has a
sample of undergraduate nursing students and which uses
the, FC method, observed significantly better scores on the
ophthalmology courses in the intervention group (Zhu et al.,
2020). A quasi-experimental study (Kim et al., 2019),
using FC in patient safety course, showed that the attitude,
skills and knowledge scores were statistically higher than
the control group. Another quasi-experimental study
(Dehghanzadeh & Jafaraghaee, 2018), using FC regarding
the critical thinking approach acquired significantly higher
scores than the TT. A study (El-Banna et al., 2017), using
a mixed methods approach, and referring to FC in
pharmacology course, showed that the first exam scores
were significantly higher in the intervention group.
Moreover, another quasi-experimental quantitative study
(Chu et al., 2019), which made use of FC in an evidence
based nursing course found that the scores of the
with the control group. One more quasi-experimental study
(Awad & El-Adham, 2019), showed significantly higher
scores regarding the community health nursing course in
the experimental group as against the control group. A
systematic review and meta-analysis of randomized
controlled trials with nursing students (Xu et al., 2019),
using FC approach showed that the flipped classroom
increased the students’ skills scores significantly when
compared with the traditional teaching method. A
systematic review and meta-analysis (Tan et al., 2017),
using FC in nursing education indicated that there was a
significant post-intervention improvement in academic
performance both in knowledge and skills. Regarding
students’ satisfaction scores, Awad and El-Adham (2019)
used FC in terms of student satisfaction and engagement;
finding significantly higher scores between pre and post-
test of the case and control group. In a quasi-experimental
study (Dehghanzadeh & Jafaraghaee, 2018), positive
experiences of the FC included improving memorization
while negative experiences included the mandatory
watching of the electronic contents while studying the
book of the course as well.
As is understood from the study findings, there have been
consistent results on the effects of the FC during recent
years. However, some studies have provided negative
results on the effects of the FC and the quantity of
empirical studies regarding the effectiveness of FC in the
“Leadership in Nursing” course are insufficient. Therefore,
the purpose of this study was to evaluate the effect of the
FC via the “Leadership in Nursing” course over nursing
students’ achievements (exam scores and learning
outcomes) and to describe students’ experiences of FC.
Course success was constituted by the overall grades
obtained from the assignment, and the midterm and final
exam scores. Both exams and the assignment were
evaluated over 100 points. The overall grade was
comprised of the sum of 30% of the midterm exam, 10%
of the assignment, and 60% of the final exam score.
Learning outcomes were constituted by the researcher as
16 general objectives; 44 cognitive sub-objectives and 8
affective sub-objectives in order to measure whether all
the students obtained their cognitive and affective learning
outcomes for the “Leadership in Nursing” course. Bloom
Taxonomic cognitive objectives had “knowledge”,
“comprehension”, “analysis”, “application” and
“synthesis” steps, whereas affective domain had the steps
of “responding” and “characterizing”. The hypotheses of
the study were as follows:
H1-1. Students educated with the FC approach will be
more successful than those in the TT group in terms of
course success.
H1-2. Students educated with the FC approach will be
more successful than those in the TT group in terms of
achieving learning outcomes.
Method
Design of the study
A quasi-experimental design was used in this study. This
quasi- experimental study compared two learning groups
taking “Leadership in Nursing”: a selective course in the
nursing curriculum. Students were enrolled in the
“Leadership in Nursing” course offered in the Department
of Nursing of the Faculty of Health Sciences of one
university in the fall semester of the 2017–2018 academic
years. The intervention group received FC, while the
comparison group experienced TT. The students in both
two groups had 2 classes per week for 14 weeks.
Participants
The study population included all 40 third-year students
who were enrolled to the “Leadership in Nursing” course.
All of the students accepted the invitation to participate
and were included in the study sample. The students were
assigned to the experimental group (n = 20) and to the
control group (n = 19) via a random number generating
process. Students who were participating in the study were
informed about the aim and method of the study and their
written informed consent was obtained. The students were
informed that they could quit the study any time they
wanted to.
Data collection tools
Data was collected through the means of; the “Student
Introductory Information Form”, “Exams (a midterm
exam, a final exam and 4 assignments)”, the “In-Class
Observation Form” and the “Student Feedback Form”.
Quantitative methods were used to assess students’
cognitive development and exam scores while interviews
were used to collect data on regarding students’
experiences of the FC.
Student Introductory Information Form
The form was prepared by the researcher in order to
describe the characteristics of the experiment and control
groups including age, sex, information regarding
graduation, internet usage status, weekly internet usage
and web based education experience (Table 2).
Exams
Exams were prepared by the researcher in order to
measure whether all of the students obtained their
specified cognitive learning outcome. Cognitive learning
objectives were evaluated via an assignment, a midterm
exam and a final exam. At first, table of specifications was
created. Then, test
questions were created with the help of the literature. After “responding” and 1 was “characterizing” of the Bloom
an expert consensus was reached with regards to the
cognitive learning objectives for the “Leadership in Taxonomy. In the evaluation of the affective domain, “the
Nursing” course, 16 general objectives and 44 cognitive number of students who raised their hands in the
sub-objectives were established. classroom” was taken as the criterion. Due to the limited
The midterm was consisted of 25 questions in total; 12 of course hours, few students were observed for affective
them formed up the “knowledge” stage, of the cognitive
field of the Bloom Taxonomy while the 4 of them domain. Student Feedback Form
constituted the “comprehension” stage and the last 9 made
The “Student Feedback Form” was prepared in order to
up the “analysis” stage. The final was consisted of 40
measure the thoughts and satisfaction levels of the students
questions in total; 23 of them formed up the “knowledge” in the FC group after taking the course. The form
stage, of the cognitive field of the Bloom Taxonomy while contained open-ended questions and students were asked
the 5 of them constituted the “comprehension” stage and to share their positive and negative opinions related to the
the last 12 made up the “analysis” stage. Achievement of FC learning experience.
cognitive domain learning objectives of the “Leadership
Concept, Nursing and Leadership Process, Leadership Received expert opinions
Theories, Historical Perspective on Nursing Leadership,
The compatibility of the items of “midterm, assignment,
Gender and Leadership” units were evaluated with the
final and in- class observation” with cognitive and
mid- term and final exam.
affective objectives was tested/ evaluated by three expert
“Influence Process: Use of Power, Weakness-
faculty members from the Department of Measurement
Empowerment, Vision Development-Visionary Leadership
and Evaluation in Education. The content validity index
and Individual and Professional Positioning” units were
was calculated using the “Davis” technique in terms of the
evaluated with the final exam. Finally, the assignment was
compatibility of the items of “midterm, assignment, final
consisted of 4 subjects; “Historical Perspective on Nursing
and in-class observation” for the “Leadership in Nursing”
Leadership and Vision Development-Visionary
course regarding the cognitive and affective goals.
Leadership” were “synthesis” whereas “Individual and
Qualitative data was tested/evaluated by three expert
Professional Positioning” were “application” steps which
faculty members from the Turkish Language and
were established of the levels of cognitive domain. The FC
Literature and analyzed by two researchers. Participants
and TT groups received the same exams and assignments.
confirmed the themes.
Students in the FC group prepared and delivered their
assignment within the time allocated to them in class. Establishing the FC teaching program
Students in the control group prepared their assignment at
home and delivered them when they came to the class. The Based on previous experiences, new teaching programs
midterm exam was held in the 6th week, the final exam were designed for FC and TT groups in the “Leadership in
was held in the 14th week and the assignment was to be Nursing” course. The course duration was 14 weeks,
submitted in the 4th, 10th, 11th, and in the12th week including 12 weeks of instruction and 2 weeks of
(Table 1). evaluation. The program consisted of 8 units. For each unit
an instructional plan was developed. PowerPoint
Evaluation of the exams presentations were prepared for each subject of the 8 units
in the program. For the FC group, recorded educational
Both of the exams and the assignment were scored on the
materials (YouTube presentations, exercises, short
scale of 100 points. The exam scores were calculated on
questions, cases, puzzles) were prepared for out-of-class
the computer and mean scores were created for each
and in-class activities (Fig. 1 and Table 1).
question which was obtained by each student. The overall
grade was comprised from the sum of the midterm exam
(30%), of the assignment (10%), and the final exam score
(%60). In-Class Observation Form

The “In-Class Observation Form” was used to evaluate

whether the learning outcomes in the affective domain
were achieved for both groups. There were 8 affective
goals in total, while 7 of them were
There were 8 unit subjects in total; “Leadership Leadership Theories, Historical Perspective on Nursing Leadership,
Concept, Nursing and Leadership Process, Gender and Leadership, Influence Process: Use of Power,
Weakness- Empowerment, Vision Development-Visionary
Leadership and Positioning”

(Table
1
Leadership concept
Say or write the definition of leadership
concept
Count the leadership items Express
the difference between leader and
follower
2 Say at least 3 of the differences between
Nursing and leadership leader and manager Determine the type
process of leader you want to be
Analyze the differences between Knowledge (CD) Video conference Q&A session
leadership styles
Willingness to participate to the Comprehension (CD) Learning exercise Learning exercise
discussion on the differences between
leader and manager Analysis (CD) Reading passage

Knowledge (CD) Video conference Youtube video

3 List the leadership theories Distinguish
Leadership theories the ways in which leadership skills are Comprehension (CD) Learning exercise Grup discussion
developed Analyze the differences
between leadership styles Analysis (CD) Reading passage Individual study
Willingness to share the leadership
styles experience Responding (AD) Reading article Word puzzles Diagramming

4 Count five of the nurse leaders in

Knowledge (CD) Video conference Quiz
Historical perspective on history
nursing leadership Explain leaderhip characteristics of Comprehension (CD) Learning exercise Grup discussion
nurse leaders Reading passage Case study
Associate Florence Nightingale’s
behavior with leadership behaviors Analysis (CD) (book chapter) Speech ring
5 Discuss strategies to become
Gender and leadership an effective female leader Responding (AD)
Explain the psychological, social and Knowledge (CD) Video conference Biography reading and writing
polithical factors that affect women report
leadership in instution
Distinguish the leadership behaviors of Comprehension (CD) Reading article Group case study
female and male leaders Synthesis (CD) (notes from Selimiye) Prepare assignmentb
Willingness to share experience with
gender and leadership
Knowledge (CD) Video conference Q&A session
6 Mid-term
Comprehension (CD) Learning exercise Diagramming
7 List at least three of the concepts
Influence process: use of with which power is related Explain
force two of the ways of developing Analysis (CD) Youtube video Group case study
power sources Identify the
fundamental relationship between Responding (AD) (Margaret Thatcher)
power and influence
Work with harmony in group activity
Video conference Group working
Learning exercise Reading Learning exercise
8 List at least six strategies for creating article (A Concept Analysis: Group activity
Influence process: use of group strength with nurses Discuss the Knowledge (CD) Power in Nursing) Q&A session
force importance of being strong in nursing Diagramming
Distinguish between strong and weak Comprehension (CD) Analysis (CD)
nurses
Participate in classroom discussions
Characterizing (AD)
Knowledge (CD) Video conference Group working
9 Define the concept of powerlessness Learning exercise Learning exercise Reading article
Powerlessness- Express how nursing looks like in the Reading article
Comprehension (CD) and discuss
empowerment society Word puzzle game
Willingness to share experience with Analysis (CD)
empowerment
Responding (AD)
Knowledge (CD) Video conference Group working
Comprehension (CD) Learning exercise Learning exercise
Count individual power phases Responding (AD) Reading article Reading article and discuss
10
Establish a relationship between Diagramming
Powerlessness-
individuals’ behavior and individual Youtube video
empowerment
empowerment

Say the definition of the vision concept Knowledge (CD) Video conference Case working
Develop an individual vision Analysis (CD) Learning exercise Determine an individual power
11
Vision development stage
Diagramming

12 Discuss the importance of visionary

Visionary leadership leadership Knowledge (CD) Video conference Learning exercise Group activity
Bloom taxonomy Synthesis (CD)
Knowledge (CD)
Knowledge (CD)
Implementation regarding the FC group
The students were randomly assigned to the experimental (FC) group
(n = 20), and they were informed about the study, the FC approach, and
how to use the Learning Management System (LMS). In the out-of-class
part of the FC program, the students studied 15–20 min of lecture on
the related subject, then, they had access to the teaching materials for
learning objectives such as PowerPoint presentations, movies, articles,
learning exercises which were used for 30 min. The course instructor
evaluated the completed learning exercises and provided feedback via
the LMS for approximately 15 min.
The in-class part of the FC program started with a 10–15 min
discussion of topics which the students did not understand from the out-
of- class material and of a Q&A session while the instructor provided
feedback on the learning exercises. This was followed by some in-class
activities as case studies and watching videos for approximately 65 min.
At the end of the class, students were asked to summarize the subject
within approximately 10 min. Observations about the students were
recorded on the observation form in the classroom. After the course, the
students completed the student feedback form which was about FC
education (Fig. 1 and Table 1).
Implementation regarding the control group
The students in the control (TT) group (n = 19) were informed that
they would take the course in the standard in-class format. Face-to-face
training was conducted for 30–40 min using PowerPoint presentations
which continued with a 15 minute-discussion about answering the
unanswered questions of the students. This was followed by some in-
class activities such as case studies, watching videos or movies, reading
biography, diagramming, group discussions, learning exercises, question-
answer discussions, solving puzzles and role playing which lasted about
75 min.
Achievement of learning objectives was assessed through the usage
of the midterm exam, the assignment and the final exam (Fig. 1). There
were 8 unit subjects in total; “Leadership Concept, Nursing and
Leadership Process, Leadership Theories, Historical Perspective on
Develop an individual vision Prepare assignmentb
Video conference Youtube video (Atatürk in
Çanakkale)
Nursing
Leadership, Gender and
Leadership, Influence
Process: Use of Power, Weakness-Empowerment, Vision Development-
Visionary Leadership and Positioning” (Table 1).
Data analysis
Quantitative data was evaluated using the SPSS statistics package
program. Descriptive statistics were given for categorical and numerical
variables. Mann–Whitney U test, chi-square test, Fisher’s exact test and
Friedman two-way analysis of variance (ANOVA) were used in the
analyses. P values <0.05 were accepted as the significance level in all
statistical tests.
In order to analyze the views of students regarding the process of
learning with the FC, codes of “Students Positive Opinions on Learning
with FC” and “Students Negative Opinions on Learning with FC” were
formed. Student views were divided into 2 sub-headings as out-of-class
and in-class learning. Similar expressions related to each learning
process were selected and categorized. The categorized meanings were
grouped under the heading of “student views”. As such, this was
presented in the results. Examples that make up each view were written
one by one (Table 4).
Ethics
The ethical approval and written permissions were obtained from
the Cankiri Karatekin University ethics committee and the Dean of
Faculty of Health Sciences. The students were informed about the study
and written informed consent was obtained.
Results
Quantitative data Sociodemographic characteristics
The mean age of the experimental group was 21.26 + 1.69; while the
mean age of the control group was 20.45 + 0.89, which were similar to
each other. 90% of the students in the experimental group and 63.16%
of
the students in the control group were women. 55% of the students in
the experimental group and 57.89% of the students in the control group
graduated from an Anatolian high school. 95% of the students in the
experimental

group and 84.21% of the students in the control group reported that reported using the internet for 1 h or less in the control group. 80% of
they used internet resources for nursing education. 65% of the students the students in the experimental group and 78.95% of the students in
reported using the internet for 2 h or more per week for nursing the control group have never received a web-based education before
education in the experimental group and % 63.16 of the students (Table 2).

Exam scores Learning outcomes

Compared to the control group, the FC group had obtained
significantly higher course assignment score (t = 1.997, p <0.05), final
exam scores (z = − 1.84, p <0.05) and a higher overall grade as a result (z
= − 1.515, p <0.05). The effect sizes of the difference between the FC and
control groups in these assessments were 0.318, 0.336, and 0.320,
respectively, indicating a large effect. The results showed that students
in the FC group were more successful than students in the TT group in
achieving course objectives; therefore, the H1-1 hypothesis was
confirmed (Table 3).
In the midterm exam, there was not a significant difference between
the two groups in terms of achieving the objectives on the levels of
knowledge, comprehension, and analysis levels of the cognitive domain
(p >0.05).
In the final exam, it was observed that students in the FC group had
significantly higher achievement rates of objectives regarding the
knowledge (z = − 2.543, p <0.05) and analysis (z = − 1.835, p <0.05) levels
of the cognitive domain compared to the control group (Table 4).
In terms of assignment scores, objective achievement in the
application number 1 (z = − 2.859, p <0.05), application number 2

Table 2
Introductory characteristics of students in the flipped classroom and control
groups.

Introductory characteristics of students in the flipped classroom and control

groups.

a
Variables While examining the relationships between groups of nominal
variables, chi- square analysis was applied. Fisher’s Exact Test Chi-Square analysis was applied in cases where the expected values in the cells in 2 ×2 tables in which did not
have sufficient volume.
Table 3
Mean scores of students in the flipped classroom and control groups.
Exam types Teaching Mean scores Statistical evaluation

method
X+SD z and t p Cohen’s
values d
Midterm exam FC TT 0.745 0.049
79.21 ± 11.39 z = − 0.325
79.81 ± 13.21
Assignment score FC TT 67.67 ± 24.5 t = 1.997 0.032 0.318
58.81 ± 29.92
Final exam FC 68.84 ± 14.84 z = 1.84 0.041 0.336
TT 62.62 ± 21.55

Overall gradea FC TT 71.74 ± 12.31 z = 1.515 0.046 0.320

66.29 + 20.7

a
Overall grade was the sum of 30% of the midterm exam, 10% of the
assignment and 60% of the final exam score.
(z = − 3.39, p <0.05), and syntheses number 2 levels (z = 2.388, p <0.05)
of the cognitive domain were found to be significantly higher in the FC
group compared to the control group. Our results showed that students
in the FC group were more successful in terms of accomplishing learning
objectives related to the “leadership in nursing” course, and the H1-2
hypothesis was confirmed (Table 5).
No significant difference was found between the two groups in terms
of performing the “responding” and “characterizing” levels of the
affective domain.
Qualitative data Interview results
Our data collection tools included open-ended questions and
positive and negative opinions of FC. Students’ opinions were divided
into two categories, out-of-class and in-class learning. Students’ positive
statements about FC were categorized into three categories, namely
 X+SD z values p Cohen’s d X+SD z values p Cohen’s d

Knowledge FC 36.9 + 6.1 − 0.055 0.956 0.092 41.4 + 10.3 − 2.543 0.043 0.494
TT 36.3 + 6.9 35.3 + 14.1

Comprehension FC 10.3 + 4.5 − 0.781 0.435 0.205 10.4 + 2.7 − 0.368 0.713 0.029
TT 11.1 + 3.2 10.3 + 4.0

Analysis FC 32.9 + 4.6 − 0.095 0.924 0.239 17.6 + 3.8 − 1.835 0.048 0.231
TT 31.4 + 7.6 16.6 + 4.8

Overall FC 80.0 + 11.4 − 0.135 0.892 0.096 69.3 + 14.8 − 2.055 0.041 0.381
TT 78.8 + 13.5 62.2 + 21.8

+ SD + SD z value p Cohen’s d

Application (1) To prepare a notification including the role and functions of a nurse leader 72.5 + 43.6 67.5 + 27.0 − 2.859 0.032 0.138

Application (2) To prepare individual positioning notification 73.0 + 38.5 43.8 + 40.5 − 3.39 0.017 0.739

Syntheses (1) To develop individual vision 70.0 + 47.0 75.0 + 44.4 − 0.35 0.727 0.109

Syntheses (2) To design a logo, image, slogan or article to make nursing brand 63.8 + 37.2 48.3 + 40.1 − 2.388 0.039 0.400

“providing flexibility”, “being prepared for the lesson” and “reinforcing
learning”. One student stated, “Having lessons out of school, being able
to learn in a comfortable environment like home or the dorm was great,
we could easily access the slides and topics whenever we wanted.”
Another student
“The videos would freeze because of system errors and internet
problems”. While another student said, “Being asked to speak in class
when I was not prepared was difficult for me.” (Table 6).
Discussion

Table 4 We found that the FC approach resulted in significantly high levels of

expressed that, “By doing research we could answer the questions in the
learning activities more easily.”
On the other hand, students’ comments on the negative aspects of
FC were categorized in the following four categories: “system
infrastructure problems”, “requirement of independent work”,
“excessive number of activities” and “active participation requirement”.
One student stated,
were also some studies that reported no difference among the two techniques (Chen et al., 2017; Maxwell & Wright, 2016; Ward et al., 2018).
achievement of course objectives and higher overall course grades. In
this case, the H1-1 hypothesis was confirmed and we can say that
students educated with the FC approach will be more successful than
those in the TT group in terms of course success. This finding is
consistent with other studies of nursing undergraduate education which
reported statistically higher exam scores and achievement rates with FC
compared to TT (Zhu et al., 2020; El-Banna et al., 2017; Kim et al., 2019;
Dehghanzadeh & Jafaraghaee, 2018; Chu et al., 2019; Awad & El-
Adham, 2019; Xu et al., 2019; Tan et al., 2017; Hayırsever & Orhan,
2018), although there

Final exam scores of the FC group regarding the Schlairet, 2017). Notwithstanding, the TT group
knowledge and analysis levels of the cognitive spent less time than the FC group outside of class
domain were significantly higher than the TT on personal study. In-class environment, students
group. In the present study, it was seen that students were expected to participate in active learning
who regularly watched videos and completed methods, bring together and apply the information
learning exercises on the online platform had they had learned out-off class
significantly better results than those who did not. Table 6
FC students had access to the course material Students’ experiences of the flipped classroom.
anytime, anywhere, for any duration and had to
take responsibility for their own learning (Green
&

Category Student views

 Out-of- class ions of the flipped classroom
Providing flexibility
Being prepared for the lesson
In-class Reinforcing learning
Negative opinions of the flipped class
Out-of- System infrastructure class problems
Requirement independent work
In-class Excessive number of activities
Active participation requirement
participate in-class studies in order to achieve high-level behaviors
(Lopes & Soares, 2018). It was thought that the factors such as longer
exposure time to active learning methods in-class, and this may be
considered as a factor which contributed to this result.
However, FC allows students to use their knowledge in practice and
to actively participate in active learning methods (Dehghanzadeh &
Jafaraghaee, 2018). By doing so, students were ready for further
learning of basic concepts in order to participate effectively in in-class
activities pertaining to the analysis level of the cognitive domain. This is
consistent with the behavioral learning theory and it is developed by
establishing a link between a certain learning stimulus and behavior and
changes in behavior occur through learning by doing, repetition and acts
of reinforcement (Gilboy et al., 2015). In similar FC studies, it was
observed that students who were more active in in-class studies learned
more effectively (Gao, 2016; Hanson, 2016), obtained higher scores at
the knowledge level (Tan et al., 2017), conceptualized the subject better
(Hanson, 2016) and academic performance was influenced by final grade
(Awad & El-Adham, 2019).
The FC group students’ assignment scores at the upper cognitive
domain levels of application and synthesis were significantly better
compared to those in the TT group. We believe this can be attributed to
also indicates that the students took on their own learning
“Having lessons out of school, being able to learn in a comfortable environment like home or
the dorm was great”, “we could easily access the slides and topics whenever we wanted”,
“watching the parts that were not understood in the videos over and over helped us to
understand the issue better”, “Instead of taking a long lecture in the lesson, we could
understand the topic for 15 min at home”, “Continuous course made it easy to remember
and
learn”.
“We worked to the topics before”, “This method allowed us to learn unknown words”, “By
doing research we could answer the questions in the learning activities more easily”, “This
method allowed us to come to the lesson with information about what to do on that day”.
“My outside-class information was reinforced with in-class activities”, “The activities were
nstructive, helped to reinforce the subject and prepared for exams”, “Activities made it easier
or me to learn because they summarized the lesson”. room
“The videos would freeze because of system errors and internet problems”, “I had trouble
entering the system due to the internet problem”, “Problems with the system were very much
and it took a lot of time”.
“I did not like to prepare homework and reports with this method”, “I did not enjoy doing
learning exercises and watching videos, I was bored”.
“In this method, activities such as group work, learning exercises were many”.
“I had difficulty expressing myself in class”, “I had difficulty doing learning exercises”, “Being
asked to speak in class when I was not prepared was difficult for me”.
their exposure to active learning and that this repeated exposure
contributed to their achievement of higher-level objectives. Anolak et al.
(2018) mentioned that FC has proven to be useful for the development
of higher-level objectives and fostering nursing leadership skills.
Consistent with our findings, Lockman et al. (2017) conducted a study to
evaluate the learning outcomes of TT and FC groups regarding pain
management and reported that the FC group attained significantly
higher scores on items involving higher-level objectives (application,
analysis, synthesis) compared to the TT group.
Our students described the positive aspects of FC as “providing
flexibility”, “helping to being prepared for the lesson” and “reinforcing
learning”. In the present study, the students were able to play, stop and
rewind videos to watch the lessons at their own pace and according to
their learning style. This is an advantage because learning is a personal
process and everyone has a different learning speed and style. In the FC,
the students have to learn the concepts and watch the lessons before
attending in-class as a prerequisite to achieve high-level learning in the
classroom environment. The students’ views regarding that FC learning
promotes “being prepared for the lesson” can be attributed to this
inherent feature of FC and is related to the flexibility it provides. This
view
responsibilities.
 Similar studies reported that students took on their own learning
responsibilities and became more proactive through the usage of the FC
(Green & Schlairet, 2017; Shiau et al., 2018). As reported by our
students, one of the positive opinion towards FC was the reinforcement
of learning. In the current study and during in-class activities, students
communicated with the academician and their classmates, participated
in group discussions and exchanged information with each other. For
this reason, it is thought that learning together improves nursing
students’ self- efficacy and reinforce their learning (Palsson et al., 2017).
Positive results were also found in a quasi-experimental study
conducted on undergraduate nursing students regarding improved
learning abilities (Zhu et al., 2020). Dehghanzadeh and Jafaraghaee
(2018), reported that FC had advantages such as student centered
learning, improved learning and retention of the learnt information.
Yacout and Shosha (2016), reported that through completing the pre-
class PowerPoint presentations and workbook exercises, students
renewed their knowledge, increased the retention of the information
which they have learned, enabling them to understand the subject at
hand better.
As a result, the FC inspired the students to learn by doing research
and engaging in group work, discussing with each other, thinking and
expressing themselves. In other words, the students improved their
abilities to discuss group activities, to share experiences regarding
leadership willingly, to work in harmony when in group activity and to
participate in classroom discussions, which are all essential elements of
leadership.
As reported by our students, one of the negative opinions of FC was
the system infrastructure problem. Dehghanzadeh and Jafaraghaee
(2018), reported that FC had problems such as lack of electronic
technicians and expensive electronic material. In a study conducted by
Yacout and Shosha (2016), students stated that watching videos were
taking too much time, that they could not get feedback while watching
the videos, and that the videos were blurred. Student feedback on the
subject has been discussed. Afterwards, technical assistance was
received by gathering with the technical team and the encountered
technical problems were solved. Despite this, the technical team
reported that other universities were also using the same system of
distant education and that this situation created excessive intensity and
burdened the system. In addition, it was assumed that the majority of
the students made their pre-class preparations 1 day before the class
and the state of being connected to the same wi-fi network in their
home or dormitory environment also created an excessive density and
load in the system. Another negative opinion of FC was that it required
independent and individualized study. Possible reasons for this opinion
include the students having to study by themselves, having difficulty
with complex topics, being accustomed to conventional learning in
traditional learning settings, having different learning styles, and not
having previous experience with web-based learning. Fraga and Harmon
(2014) pointed out that the students’ learning styles may be important
at this stage. Bishop and Verleger (2013) stated that applying the lesson
in a traditional classroom environment with face-to-face lecture in a
comfortably and effortlessly way created a resistance against FC. On the
other hand, more visuals, videos and PowerPoint presentations were
prepared in an enjoyable and effective way regarding the FC. In the
lectures, students demanded more and more high quality visual
materials and video lessons, further development of these materials
(82), and they demanded the rigorous preparation of educational
material (Phillips & Trainor, 2014). Thus, it was stated that this teaching
strategy could appeal to all students who possess different learning
styles (Galway et al., 2014). Other negative opinions of FC were
“excessive number of activities” and “the requirement of active
participation”. As can be seen from the findings,
the third-year students which were receiving education in a university in
Turkey and who were registered in nursing education. Therefore, we
the educator encourages the student to think, speak and give him/her
the opportunity to express himself/herself. While taking responsibility
for learning enables students to obtain the skill of “self-control”, this
type of teaching also improves their learning skills and give them the
habit of lifelong learning (Hawks, 2014; Towle & Breda, 2014). However,
it is understood from the statements of the students that they did not
like to make preparations outside the classroom and to participate in
classroom activities. Traditional learning environments are comfortable
for students. Teaching activities conducted in these environments do not
require high level cognitive processes, and the students have their own
“rooted” learning habits. The factors such as the existence of these
habits and preparing the course materials prevent the students from
thinking, studying and fulfilling the assignment which they were tasked
with. This situation causes students to take responsibility in learning in
an unwillingly fashion. In this regard, the teaching staff should be
encouraged to use innovative learning methods which require active
participation. The requirement of active participation improves students’
achievements when they participate in activities such as group work
(Awad & El-Adham, 2019). In similar studies, El-Banna et al. (2017),
stated that when students had to learn the medical terminology they
perceived this as an overwhelming task and that the workload was too
much. Dehghanzadeh and Jafaraghaee (2018), reported that FC had
disadvantages such as its activities being time-consuming. Similarly, in
other studies, Yacout and Shosha (2016) reported that students’ most
common complaints were about the heavy workload and being exposed
to the repetition in the classroom of what they had already learned
before.
As it can be understood from these findings, educators encourage
students to think and talk while giving them an opportunity to express
themselves. They encourage the students to be proactive in classroom
discussions, to participate in activities and assume responsibility.
Students taking responsibility for their education promotes self-
confidence, while also improving their learning skills and encouraging
lifelong learning habits (Towle & Breda, 2014).
Study limitations
Several limitations of this study must be acknowledged. First of all,
the study did not address high-level cognitive objectives according to
Bloom’s taxonomy. Secondly, infrastructure problems such as difficulties
regarding access due to slow internet, long waiting periods in terms of
opening videos and pauses in the LMS were experienced during
implementation of the FC. Students in the FC group also had to learn
how to use the system in the first two weeks of the course. Thirdly, the
TT group spent less time than the FC group outside of class on personal
study. Fourthly, the conducted study was limited to third-year students
which were receiving education in a university in Turkey and who were
registered in nursing education. Fifthly, the students in the traditional
teaching group had no opportunity to complete the form that would
have provided rich information about their experiences. Other factors
which may have caused bias were that the interaction between the two
groups could not be controlled, and the fact that no observer other than
the researcher was used during data collection. Finally, due to the
limited course hours, few students were observed for affective domain
comes to the fore as a limitation.
Conclusion
In this study, we evaluated the effect of the FC approach on student
achievement in the “Leadership in Nursing” course and determined that
students studying in the FC environment had statistically significant
higher exam scores and that they demonstrated a more effective
learning process of the course objectives. But, our findings can only be
applied to
recommend further studies with different sample groups about the FC approach
should be conducted regarding the “Leadership in Nursing” courses.
 FC enables students to participate in active learning activities and it
improves their abilities in areas such as exhibiting leadership behavior,
distinguishing leadership skills, visioning, developing a new perspective,
participating in in-class discussions, willingly sharing leadership
experiences, discussing group activities, working in harmony in group
activities and participating in classroom discussions, all of which are all
essential elements of leadership skills. Therefore, we recommend
integrating the FC approach into “Leadership in Nursing” courses in
order to train nurses with strong leadership skills.
References
Al-Dossary, R. N. (2017). Leadership in nursing. In: A. Alvinius (Eds), Contemporary
leadership challenges (pp. 251).
Anolak, H., Coleman, A., & Sugden, P. (2018). Is the “flipped” pedagogical model the
answer to the challenges of rural nursing education?: A discussion paper? Nurse
Education Today, 66, 15–18. https://doi.org/10.1016/j.nedt.2018.03.026.
Awad, L. A., & El-Adham, N. A. (2019). Flipped classroom as a teaching strategy and
community health nursing student’s engagement, satisfaction and achievement.
American Journal of Nursing Research, 7(3), 362–367. DOI:10.12691/ajnr-7-3-16.
Bergmann, J., Sams, A. (2012). Flip your classroom reach every student in every class
every day. Washington USA. Part 3, Why you should flip your classroom (pp. 24).
Betihavas, V., Bridgman, H., Kornhaber, R., & Cross, M. (2016). The evidence for ‘flipping
out’: A systematic review of the flipped classroom in nursing education.
Nurse Education Today, 38, 15–21.
Bishop, J. L., & Verleger, M. A. (2013). The flipped classrom: a survey of the research.
120th ASEE annual conference & exposition. American Society for Engineering
Education; 23–26 January: 1–18.
Chen, F., Lui, A., & Martinelli, S. M. (2017). A systematic review of the effectiveness of
flipped classrooms in medical education. Medical Education, 51(6), 585–597. https://
doi.org/10.1111/medu.13272.
Chu, T. L., Wang, J., Monrouxe, L., Sung, Y. C., Kuo, C., Ho, L. H., et al. (2019). The effect
so the flipped classroom in teaching evidence based nursing: A quasi-experimental
study. PLoS One, 14(1), 1–12. https://doi.org/10.1371/journal. pone.0210606.
Cummings, G. G., MacGregor, T., Dawey, M., Lee, H., Wong, C. A., Lo, E., et al. (2010).
Leadership styles and outcome patterns for the nursing workforce and work
environment: A systematic review. International Journal of Nursin Studies, 47(3),
363–385. https://doi.org/10.1016/j.ijnurstu.2009.08.006.
Curtis, E., Vries, J., & Sheerin, F. (2011). Developing leadership in nursing: Exploring core
factors. British Journal of Nursing, 20(5), 306–309. doi:10.12968/bjon.2011
.20.5.306.
Dehghanzadeh, S., & Jafaraghaee, F. (2018). Comparing the effects of traditional lecture
and flipped classroom on nursing students’ critical thinking disposition: A quasi-
experimental study. Nurse Education Today, 71, 151–156. https://doi.org/10.1016/j.
nedt.2018.09.027.
Demeh, W., & Rosengren, K. (2015). The visualisation of clinical leadership in the content
of nursing education—A qualitative study of nursing students’ experiences. Nurse
Education Today, 35, 888–893. https://doi.org/10.1016/j.nedt.2015.02.020.
El-Banna, M. M., Whitlow, M., & McNelis, A. M. (2017). Flipping around the classroom:
Accelerated bachelor of science in nursing students’ satisfaction and achievement.
Nurse Education Today, 56, 41–46. https://doi.org/10.1016/j.nedt.2017.06.003.
Fraga, L. M., & Harmon, J. (2014). The flipped classroom model of learning in higher
education: An investigation of preservice teachers’ perspectives and achievement.
J Digital Learning in Teacher Education, 31(1), 18–27.
Galway, L. P., Corbett, K. K., Takaro, T. T., Tairyan, K., & Frank, E. (2014). A novel
integration of online and flipped classroom instructional models in public health
higher education. BMC Medical Education, 14(181), 1–9.
Gao, Y. (2016). Flipped teaching with mobile teaching platform for fundamental nursing
skill. Journal of Nursing, 23(8), 13–15.
Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using
the flipped classroom. Journal of Nutrition Education and Behavior, 47(1), 109–114.
https://doi.org/10.1016/j.jneb.2014.08.008.
Gillispie, V. (2016). Using the flipped classroom to bridge the gap to generation. The
Ochsner Journal, 16(1), 32–36.
Green, R. D., & Schlairet, M. C. (2017). Moving toward heutagogical learning:
Illuminating undergraduate nursing students’ experiences in a flipped classroom.
Nurse Education Today, 49, 122–128. https://doi.org/10.1016/j.nedt.2016.11.016.
Hanson, J. (2016). Surveying the experiences and perceptions of undergraduate nursing
students of a flipped classroom approach to increase understanding of drug science
and its application to clinical practice. Nurse Education in Practise, 16(1), 79–85.
https://doi.org/10.1016/j.nepr.2015.09.001.
Hawks, S. J. (2014). The flipped classroom: Now or never? Education News., 82(4),
264–269.
Hayırsever, F., & Orhan, A. (2018). A theoretical analysis of flipped learning. Mersin
University Journal of the Faculty of Education, 14(2), 572–596. doi:10.17860/me
rsinefd.431745.
Hessler, K. L. (2016). Nursing education: Flipping the classroom. Nurse Practitioner, 41
(2), 17–27.
Kim, Y. M., Yoon, Y. S., Hong, H. C., & Min, A. (2019). Effects of a patient safety course
using a flipped classroom approach among undergraduate nursing students: A
quasi- experimental study. Nurse Education Today, 79, 180–187.
https://doi.org/10.1016/j. nedt.2019.05.033.
Kling, V. (2010). Clinical leadership project. Journal of Nursing Education, 49(11), 640–
643. https://doi.org/10.3928/01484834-20100831-02.
Lockman, K., Haines, S. T., & McPherson, M. L. (2017). Improved learning outcomes after
flipping a therapeutics module: Results of a controlled trial. Academic Medicine, 92
(12), 1786–1793.
Lopes, P. A., & Soares, F. (2018). Perception and performance in a flipped financial
mathematics classroom. International Journal of Management in Education, 16(1)
(105-103).
Maxwell, K. L., & Wright, V. H. (2016). Evaluating the effectiveness of two teaching
strategies to improve nursing students’ knowledge, skills and attitudes about quality
improvement and patient safety. Nursing Education Perspectives, 37(5), 291–292.
Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping the
classroom to improve student performance and satisfaction. The Journal of Nursing
Education, 52(10), 597599.
Mortensen, C. J., & Nicholson, A. M. (2015). The flipped classroom stimulates greater
learning and is a modern 21st century approach to teaching today’s
undergraduates. Journal of Animal Science, 93(7), 3722–3731.
National Academy of Medicine. (2010). The future of nursing; leading change, advancing
health, summary report. http://www.nationalacademies.org/hmd/Reports/2010
/The-Future-of-Nursing-Leading-Change-Advancing-Health.aspx. (Accessed 18
August 2016).
Palsson, Y., Martensson, G., Swenne, C. L., Adel, E., & Engstrom, M. (2017). A peer ¨
learning intervention for nursing students in clinical practice education: A quasi-
experimental study. Nurse Education Today, 51, 81–87. https://doi.org/10.1016/j.
nedt.2017.01.011.
Phillips, C. R., & Trainor, J. E. (2014). Millennial students and the flipped classroom.
Proceedins of ASBBS, 21(1), 519–530.
Richardson, V. (2003). Constructivist pedagogy. Teachers College Record, 105(9),
16231640.
Sherbino, J., Chan, T., & Schiff, K. (2013). The reverse classroom: Lectures on your own
and homework with faculty. CJEM, 15(3), 178–180.
Shiau, S., Kahn, L. G., Platt, J., Li, C., Guzman, J. T., Kornhauser, Z. G., et al. (2018).
Evaluation of a flipped classroom approach to learning introductory epidemiology.
BMC Medical Education, 18(1), 63.
Symenuk, P., & Godberson, S. (2018). Leadership experiences in undergraduate nursing
education: A student perspective. International Journal of Nursing Student
Scholarship,
5(29), 1–6.
Tan, C., Yue, W. G., & Fu, Y. (2017). Effectiveness of flipped classrooms in nursing
education: Systematic review and meta-analysis. Chinese Nursing Research, 4(4),
192–200. https://doi.org/10.1016/j.cnre.2017.10.006.
Towle, A., & Breda, K. (2014). Teaching the millennial nursing student: Using a “flipping
the classroom” model. Nursing and Health, 2(6), 107–114.
Ward, M., Knowlton, M. C., & Laney, C. W. (2018). The flip side of traditional nursing
education: A literature review. Nurse Education in Practice, 29, 163–171.
https://doi. org/10.1016/j.nepr.2018.01.003.
World Health Organization (2016). Nurse educator core competencies. https://www.wh
o.int/hrh/nursing_midwifery/nurse_educator050416.pdf Accessed 16 August 2018.
Xu, P., Chen, Y., Nie, W., Wang, Y., Song, T., Li, H., et al. (2019). The effectiveness of a
flipped classroom on the development of Chinese nursing students’ skill
competence: A systematic review and meta-analysis. Nurse Education Today, 80,
67–77. https:// doi.org/10.1016/j.nedt.2019.06.005.
Yacout, D. S., & Shosha, A. F. (2016). Nursing students’ perceptions towards flipped
classroom educational strategy. Journal of American Science, 12(2), 62–75.
Zhu, L., Lian, Z., & Engstrom, M. (2020). Use of a flipped classroom in ophthalmology ¨
courses for nursing, dental and medical students: A quasi-experimental study using
a mixed-methods approach. Nurse education Today, 85, 104262. https://doi.org/
10.1016/j.nedt.2019.104262.
 Nurse Education Today 97 (2021) 104710

Promoting engagement of nursing students in online learning: Use of the student-generated quest
a nursing leadership course
Samantha Juan
10-4238 Bond St, Burnaby, BC V5H1G4, Canada

ARTICLEINFO ABSTRACT
Keywords: Background: The frequency of online learning has surged in nursing education, and engagement is a critical element in
Nursing students students’ success in online learning. An evidence-based learning tool is needed to promote engagement in online nursing
Engagement education.
Online learning
Objectives: To examine the effectiveness of the student-generated question (SGQ) in promoting engagement in an online
Student-generated question (SGQ)
Leadership course nursing leadership course.
Design: A quasi-experimental and post-test design.
Setting: The online nursing leadership course at a university in Canada.
Participants: 179 undergraduate nursing students.
Methods: Six learning behaviors related to engagement, including number of visits, time spent, topics visited, posts read,
threads created, and replies posted, were retrieved from the institution’s learning management system (LMS). The equality
of covariance matrices was applied to examine the homogeneity of the six outcome variables among the control and
intervention groups, and multivariate analysis of variance (MANOVA) was conducted to compare the statistical differences in
all six outcome variables for three planned phases of instruction.
Results: The equality of the covariance matrices indicated that the six outcome variables were homogeneous across the
control and intervention groups. The multivariate test showed a statistically significant difference between the control and
intervention groups in the post-test measurements. In addition, the MANOVA results demonstrated statistically significant
differences in the two phases when SGQ was implemented but was statistically insignificant when SGQ was not
implemented. The intervention group demonstrated higher means in the two phases in which SGQ was implemented.
Conclusions: The findings support SGQ as an effective learning tool for promoting engagement in an online nursing
leadership course.

Introduction online due to the social distancing guidelines associated

The frequency of online learning in higher education has
surged, including in nursing education. In Canada, the
number of nursing programs offered entirely online or in
blended models of learning has increased from 62 in 2007
to 109 in 2018, and more than 55% of all programs across
Canada have partial or fully online learning (Canadian
Nurses Association, 2008; Canadian Nurses Association,
2019). In the United States, the number of online nursing
education programs has grown at a faster rate, from 96 in
2007 (Kolowich, 2010) to more than 600 in 2019
(American Association of Colleges of Nursing, 2019).
Meanwhile, many nursing programs have rapidly
transitioned theoretical courses and clinical practicum
E-mail address: Samantha_Juan@bcit.ca. https://doi.org/10.1016/j.nedt.2020.104710
with the 2020 pandemic.
Online learning has significant benefits and challenges,
which are shared among students and faculty. The shared
benefits include flexibility, convenience, and adult-
oriented learning (Billings et al., 2019, p.98). However,
student retention in online courses is 10–50% less than
that of the comparable face-to-face courses (Cameron et
al., 2011; Herbert, 2006). Several learning models have
been developed, and a variety of evidence has been
produced, to explore the issue of online learning retention
and provide solutions; examples include the Community
of Inquiry model (Garrison et al., 2010), Connecting CoL,
Learning, and Retention (Meyer, 2014), and Model for
Sustainable
R ived 1 September 2020; Received in revised form 17 November 2020;
e Accepted 2 December 2020 Available online 11 December
c 2020
e 0260-6917/© 2020 Elsevier Ltd. All rights reserved.

Student Retention (Berge and Huang, 2004). Although the In addition, the analytical data from the LMS can be
factors related to student dropout are varied, nonetheless used to assess emotional engagement. Black et
those models and evidence all support the use of one
possible solution, engagement, to promote online learning
retention.
It is evident (Tinto, 1998; Bean and Metzner, 1985) that
engagement is the key factor for the retention and
completion of a degree in campus- based higher
education. The more that students interact with faculty
and participate in academic and campus activities, the
more that they develop their confidence and skills in
general, as well as their specific confidence in learning.
Engagement not only enhances student retention in
campus-based education but also in online education,
although the interaction between students and faculty in
online learning usually involves technology.
A variety of conceptualizations of engagement
exists. However, cognitive and emotional
engagements have both

Background
Learning management system (LMS)
A learning management system (LMS), or course
management system, is a web-based system for
delivering and tracking online learning. Instructors
can share course materials, make announcements,
review assignments, administer exams, and
communicate with students both asynchronously
and synchronously through the course’s LMS. The
LMS also provides interactive tools for online
engagement, such as a discussion board, chat
room, and gamification, which aligns with
constructivist approaches (Rovai, 2004) by
enhancing knowledge construction in the
interactive learning process rather than the simple
transmission of knowledge. LMS data as a
measurement tool
Meanwhile, the LMS can be used to track learning
progress and identify learning gaps by utilizing
analytical data and reporting systems. Purdue
University implemented the Signals project
(Arnold, 2010) by applying data analytics from the
LMS to identify students who are at-risk and
provide appropriate support to meet their learning
needs, ultimately to improve institutional retention
and graduation rates. You (2016) tracked 530
undergraduate students learning behaviors in an
online course from the LMS, including virtual
attendance score, total time spent, frequency of
accessing the online course, proof of reading the
course information, and the messages created,
with the findings revealing that those behaviors
were predictors of course achievement.
al. (2008) recruited 67 graduate students
from online courses and revealed that the
data logs, including the number of page
visits, posts, and posts read, had a
significant strong positive relationship with
the sense of classroom community and
feelings of connectedness. In Henrie,
Bodily, Larsen & Graham’s study (2018),
the LMS log data, including course pages
visited, time spent on each course page,
number of discussion posts and replies
created, and self- reported engagement
survey from 153 undergraduate students in
a blended course were analyzed to examine
the correlation. There was no statistically
significant correlation between students’
self-reported engagement in the study;
however, the scatterplots from the results
indicated possible unobserved variables
that might explain the relationship between
the log data and students’ self-reported
engagement.
The student-generated question (SGQ)
The student-generated question (SGQ) is a
learning tool that can be used to improve
academic performance and engagement.
Jobs et al. (2013) recruited 102 fourth-year
medical students, providing a manual for
multiple choice question (MCQ)
intervention groups experienced a similar
academic calendar and had the same three online
course instructors. The learning behaviors were
retrieved from the LMS, Desire2Learn®, as
measures that can be tracked without the
participants’ awareness to avoid the threat of
reactivity to the study situation (Polit and Beck,
2008).
Participants
The participants were 179 undergraduate nursing
students from two nursing leadership courses at a
university in Canada. The study included 88
participants from the 2017 winter cohort in the
control group, although one participant withdrew
from the course before completion, which led to
87 participants in the control group. The
intervention group included 92 participants from
the 2017 fall cohort.
Intervention
The nursing leadership course was a 100%-online,
16-week-long course with 12 modules spanning
five of six terms in the nursing program. In the
original instructional design of the course, three
written exams were administered at weeks six, 11,
and 16, for which Honorlock® was activated and
the students had no access to the LMS course
content; online discussions at weeks two, four,
development from which a pool of 300 high-
quality MCQs were developed from the SGQ
activity. The lower-performing students from this
study demonstrated a significant improvement in
exam results compared to the higher-performing
students. Meanwhile, Pittenger and Lounsbery
(2011) applied SGQ in an online pharmacy
orientation course, with the 89 participants
exhibiting a greater degree of content engagement
in SGQ. Through the construction of question
writing, students have the opportunity to gain a
deeper understanding of the course content and
take a more active role in their learning. However,
question writing requires additional instruction to
generate questions that are at the level of
application and higher, especially for multiple
choice questions that involve meaningful
application.
Because engagement is one of the key factors for
the retention and completion of online learning
programs, it is critical to identify evidence-based
learning tools that promote engagement. The
purpose of this study was to examine whether
SGQ could improve students’ engagement in an
online nursing course.
Methods
Design
A quasi-experimental and post-test design was
employed in the study. Both control and
eight, nine, 12, and 14; and no learning activity at weeks three, five,
seven, 10, 13, and 15 (Table 1: Comparison of course schedules for
the control and intervention groups). Students were requested to post
one initial discussion thread and to reply to at least one peer’s
discussion thread before the assignment due date. Each online
discussion thread was assessed by a course-related rubric with
designated points. Each of the written exams was weighted to 25% of
the total grade, and the six online discussions were worth 25% of the
total course grade.
The intervention was to implement SGQ activities in weeks three,
five, 13, and 15 (Table 1) in the course. The unfolding learning
activities in relation to Bloom’s Taxonomy and the effective
approaches been supported and established by
for writing multiple choice
question items (McDonald,
2017) were embedded in the
module works. The
participants in the intervention
group were requested to
generate one multiple choice
question (MCQ) at the
knowledge level and two
MCQs at the comprehension
level in week three, one MCQ
for each cognitive level of
knowledge, comprehension,
Table 1
Comparison of course
schedules for the control and
intervention groups.

Week Module Control Intervention

Group Group
Week Introduction Introduction: Introduction:
1 non-graded non-graded
Week Module 1 Module 1 Module 1
2 discussion: discussion:
20 points 20 points
Week Module 2 No Activity Module 2:
3 SGQs 20
point
Week Module 3 Module 2 No Activity
4 and 3
discussion:
40 points
Week Module 4 No Activity Module 3 +
5 4: SGQs 20
point
Week Exam 1 Exam 1
6
Week Module 5 No Activity No Activity
7
Week Module 6 Module 5 Module 5
8 and 6 and 6
discussion: discussion:
40 points 40 points
Week Module 7 Module 7 Module 7
9 discussion: discussion:
20 points 20 points
Week 10 Module 8 No Activity
Week Exam 2
11
Week 12 Module 9 Module 9 discussion: 20 points
Week 13 Module 10 No Activity
Week 14 Module 11 Module 10 and 11 discussion:
40 points
Week 15 Module 12 No Activity
Week Exam 3
16
and application in week five, two MCQs at the comprehension level and
one MCQ at the application level at week 13, and one MCQ at the
comprehension level and two MCQs at the application level at week 15.
Each SGQ activity was assessed by a rubric with 20 points per activity.
The total marks for the SGQ activities were weighted to 25% of the
course grade.
Meanwhile, the instructional design of exam one relative to exam
two, from week seven to ten, remained the same for both the control
and intervention groups to explore the impact of the SGQ activity on
further learning behaviors without implementation of the SGQ activity.
Measures
The learning behaviors, including number of visits, time spent, topics
visited, posts read, threads created, and replies posted, were retrieved
and collected from the institution’s LMS, Desire2Learn®, through class
progress. The time spent was measured in seconds for the purpose of
statistical analysis. Although it would in general be possible for
participants to visit a topic and leave it open without being actively
engaged, nonetheless in this study, Desire2Learn® forced the user to
logout after 5 min of inactivity. All behaviors were measured at the
interval level.
Data collection
The data were retrieved and collected from the institution’s LMS
after all of the participants had graduated in 2018.
Ethical consideration
The study was approved by the Institute Research Ethics Board (REB
ID #2019–22). No identifiable information was collected in the study.
Data analysis
IBM SPSS Statistics version 26 was used to analyze the homogeneity
of the six outcome variables among both groups. Meanwhile, a
multivariate analysis of variance (MANOVA) was applied to compare the
difference in all six outcome variables in both groups. In addition, a
MANOVA was also conducted in three planned phases, weeks one to
five, weeks seven to ten, and weeks 12 to 15, to test the hypothesis of
the impact of SGQ activities on further learning behaviors.
In addition, the MANOVA results in Table 5 showed statistically
significant differences in terms of the number of visits (F (1, 177) =
No Activity
Exam 2
Module 9 discussion: 20 points
Module 10: SGQs 20 point
No Activity
Module 11 + 12: SGQs 20
point
Exam 3
Results
Homogeneity of variances
The equality of covariance matrices (Hahs-Vaughn, 2016) presented
in Table 2 showed no statistically significant differences (F(21,
114,464) = 1.50, p = .065; Box’s M 32.75), which indicated that the six
outcome variables were homogeneous across the control and
intervention groups, allowing MANOVA to be performed. However, the
Levene’s test of equality of error variances, as presented in Table 3,
showed statistically significant differences in regard to the number of
visits (Levene Statistic = 5.3; p = .023) and posts read (Levene Statistic =
28.23; p < .000), which indicated that those two outcome variables
might have an inequality among the control and intervention groups.
Outcome variables
The multivariate test presented in Table 4 showed a statistically
significant difference (F (6, 172) = 200.77, p < .000; Wilk’s Λ = 0.125; ηp2
= 0.875), which indicated that the post-test measurements of the
outcome variables were different from those of the control and
intervention groups. Furthermore, the number of visits (F(1, 177) =
6.714, p = .010), topics visited (F(1, 177) = 24.109, p < .00), threads
created (F (1, 177) = 1202.660, p < .00), and replies posted (F(1, 177) =
79.727, p <
.00) had statistically significant differences between the control and
intervention groups, as shown in Table 5.
The means and standard deviations (SDs) of the six outcome
variables of the control and intervention groups are presented in Table
5. The intervention group demonstrated higher mean scores in the
number of visits (M = 375.35, SD =161.11), topics visited (M = 142.57,
SD = 15.21), threads created (M = 9.05, SD = 0.43), and replies posted
(M = 8.73, SD = 1.57) in comparison to the control group in terms of the
number of visits (M = 317.79, SD =133.95), topics visited (M = 131.51,
SD = 14.91), threads created (M = 7.05, SD = 0.34), and replies posted
(M = 6.83, SD = 1.25).
Meanwhile, the multivariate test was conducted for the three
planned phases, weeks one to five, weeks seven to ten, and weeks 12 to
15, as shown in Table 4. Although all three phases demonstrated
statistically significant differences, both weeks one to five (F (6, 172) =
49.38, p < .000; Wilk’s Λ = 0.367; ηp2 = 0.633) and weeks 12 to 15 (F (6,
172) = 65.95, p < .000; Wilk’s Λ = 0.303; ηp2 = 0.697) had higher partial
eta squared (ηp2) values than the weeks seven to ten group (F (6, 172) =
7.39, p < .000; Wilk’s Λ = 0.795; ηp2 = 0.205), which indicated a larger
effect size (Richardson, 2011) on weeks one to five and 12 to 15.
11.63, p < .001.), time spent (F (1, 177) = 5.44, p = .21.), topics visited
(F (1, 177) = 8.31, p = .004), threads created (F (1, 177) =
299.804, p < .000), and replies posted (F (1, 177) = 56.67, p < < .000) and replies posted (F (1, 177) = 11.62, p = .001) in
.000) in weeks one to five, for posts read (F (1, 177) = 31.18, p weeks seven to ten, and in terms of the number of visits (F (1,
177) = 62.30, p < .000), time spent (F (1, 177) =

Multivariate test.
Value F Hypothesis df Error df Sig. Partial Eta Squared

Overall Wilks’ Lambda 0.125 200.771 6.000 172.000 0.000* 0.875

Weeks 1–5 Wilks’ Lambda 0.367 49.377 6.000 172.000 0.000* 0.633

Weeks 7–10 Wilks’ Lambda 0.795 7.392 6.000 172.000 0.000* 0.205

Weeks 12–15 Wilks’ Lambda 0.303 65.951 6.000 172.000 0.000* 0.697

*
p < .05.

Table 5
MANOVA results with m
s for the six outco
Outcome Variable M SD df Mean Square F Sig.

Overall Numbers of Visit Control Group 317.79 133.95 1 148120.776 6.714 0.010*
Intervention Group 375.35 161.11

Time Spent (Seconds) Control Group 42745.85 33079.41 1 2366911426.569 1.841 0.177*

Intervention Group 50021.37 38289.14

Topics Visited Control Group 131.51 14.91 1 5469.186 24.109 0.000*

Intervention Group 142.57 15.21

Posts Read Control Group 89.49 31.35 1 185.673 0.136 0.713*

Intervention Group 87.46 41.60

Thread Created Control Group 7.05 .34 1 180.361 1202.660 0.000*

Intervention Group 9.05 .43

Replied Posted Control Group 6.83 1.25 1 161.536 79.727 0.000*

Intervention Group 8.73 1.57

Week 1-5 Numbers of Visit Control Group 122.61 62.254 1 50997.401 11.627 0.001*
Intervention Group 156.38 69.774

Time Spent (Seconds) Control Group 15376.18 12564.379 1 907415272.702 5.439 0.021*

Intervention Group 19880.99 13240.462

Topic Visited Control Group 54.78 16.932 1 2465.112 8.312 0.004*

Intervention Group 62.21 17.489

Posts Read Control Group 39.77 13.756 1 62.939 .222 0.638*

Intervention Group 40.96 19.325

Thread Created Control Group 3.05 .301 1 46.452 299.804 0.000*

Intervention Group 4.07 .464

Replied Posted Control Group 2.41 .800 1 53.818 56.670 0.000*

Intervention Group 3.51 1.114

Week 7-10 Numbers of Visit Control Group 129.09 62.246 1 1725.241 .450 0.5038
Intervention Group 122.88 61.601

Time Spent (Seconds) Control Group 17512.21 15304.655 1 192552674.641 .981 0.323*

Intervention Group 15437.07 12663.138

 Topic Visited Control Group 47.09 10.200 1 1.800 .014 0.905*

Intervention Group 46.89 12.014

Posts Read Control Group 28.95 12.314 1 4070.427 31.183 0.000*

Intervention Group 19.41 10.516

Thread Created Control Group 2.11 .387 1 .002 .012 0.913*

Intervention Group 2.11 .377

Replied Posted Control Group 2.45 .832 1 5.492 11.622 0.001*

Intervention Group 2.10 .515

Week 12-15 Numbers of Visit Control Group 33.15 19.657 1 57018.528 65.293 0.000*
Intervention Group 68.86 36.516

Time Spent (Seconds) Control Group 5953.84 6986.197 1 961659876.816 11.502 0.001*

Intervention Group 10591.34 10793.520

Topic Visited Control Group 20.53 12.085 1 3607.556 28.554 0.000*

Intervention Group 29.51 10.379

Posts Read Control Group 20.77 8.910 1 893.054 6.309 0.013*

Intervention Group 25.24 14.153

Thread Created Control Group 1.89 .321 1 44.303 295.514 0.000*

Intervention Group 2.88 .441

Replied Posted Control Group 1.97 .493 1 56.235 141.787 0.000*

Intervention Group 3.09 .736

25.24, SD = 14.15), threads created (M = 2.88, SD = 0.44), and replies
posted (M = 3.09, SD = 0.74) in weeks 12 to 15. However, the means of
posts read (M = 19.41, SD = 10.52) and replies posted (M = 2.10, SD =
0.52) from the intervention group were smaller than those from the
control group, and similarly for posts read (M = 28.95, SD = 12.31) and
replies posted (M = 2.45, SD = 0.83), in weeks seven to ten.
Discussion
The Levene’s Test of Equality of Error Variances, as shown in Table 3,
indicated that the numbers of visits and posts read might indicate
inequality among the control and intervention groups. However, the
Equality of Covariance Matrices (Table 2) revealed statistically
nonsignificant differences among both groups prior to the intervention,
and the Multivariate Test (Table 4) showed a statistically significant
difference between the two groups in the post-test measurements.
Additionally, the results in Table 5 revealed a statistically significant
Conclusion
This study provided empirical evidence in the support of SGQ as an
effective learning tool for promoting engagement in an online nursing
leadership course. In conjunction with the literature on the use of LMS
data in assessing online engagement, the nursing education community
may benefit from the implementation of SGQ with additional
instructions related to Bloom’s Taxonomy and question writing in online
difference among weeks one to five and 12 to 15 when SGQ was
implemented but was statistically nonsignificant for weeks seven to ten
when SGQ was not implemented. The intervention group demonstrated
higher means in all of the outcome variables in weeks one to five and 12
to 15 but not weeks seven to ten. This analysis indicated that SGQ was
an effective learning tool for promoting engagement in the online
learning environment.
This finding adds more evidence to the previous studies in the
support of LMS data as an assessment tool toward the prediction of
academic performance (Arnold, 2010; You, 2016) and online
engagement (Black et al., 2008; Henrie et al., 2018) as well as SGQ as a
validated tool in improving academic performance and engagement
(Jobs et al., 2013; Pittenger and Lounsbery, 2011). However, logistical
consideration is needed when implementing SGQ, such as extra
instruction in relation to Bloom’s Taxonomy and multiple choice
question writing, to provide adequate support for students in the
development of SGQs.
courses, and students’ engagement can be tracked through the LMS to detect
the students who need additional support as early as possible.
Limitations
A convenience sample using participants from one nursing program limits
the generalization of this study’s results. Another limitation was a lack of
participant demographic data from the institution’s LMS. In line with this, the
impact on engagement by demographic characteristics, such as age and
years of experience in online learning, could not be assessed. Third, the
participants’ online learning experience in relation to SGQ was not
collected.
CRediT authorship contribution statement
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to
influence the work reported in this paper.
Acknowledgments
The author acknowledges Julie Novakovic and Kimberly Lee for their
support of this study, including the implementation of this project and
ethical application process.
framework: a retrospective. Internet High. Educ. 13 (1), 5–9. https://doi.
org/10.1016/j.iheduc.2009.10.0.
Hahs-Vaughn, D., 2016. Applied Multivariate Statistical Concepts. Taylor & Francis.
Henrie, C.R., Bodily, R., Larsen, R., Graham, C.R., 2018. Exploring the potential of LMS log
data as a proxy measure of student engagement. J. Comput. High. Educ. 30 (2), 344–
362. https://doi.org/10.1007/s12528-017-9161-1.
Herbert, M., 2006. Staying the course: a study in online student satisfaction and
retention. Online Journal of Distance Learning Administration 9 (4), 300–317.
Jobs, A., Twesten, C., Gobel, A., Bonnemeier, H., Lehnert, H., Weitz, G., 2013. Question- ¨
writing as a learning tool for students–outcomes from curricular exams. BMC
medical education 13 (1), 89. https://doi.org/10.1186/1472-6920-13-89.
Kolowich, S., 2010. Online Cure for the Nursing Crisis. Inside Higher Ed. https://www.in
sidehighered.com/news/2010/02/02/online-cure-nursing-crisis.
McDonald, M.E., 2017. The Nurse educator’s Guide to Assessing Learning Outcomes
(Jones & Bartlett Learning).
Meyer, K. A. (2014). Student Engagement Online: What Works and Why: ASHE Higher
Education Report, Volume 40, Number 6. John Wiley & Sons.
Author statement
The author, Samantha Juan, is the sole author of the project and
manuscript.
References
American Association of Colleges of Nursing. (2019). Degree completion programs for
registered nurses: RN to master’s degree and RN to baccalaureate programs. http://
www.aacnnursing.org/Portals/42/News/Factsheets/Degree-Completion-Factsheet.
pdf.
Arnold, K.E., 2010. Signals: applying academic analytics. Educ. Q. 33 (1), n1.
Bean, J.P., Metzner, B.S., 1985. A conceptual model of nontraditional undergraduate
student attrition. Rev. Educ. Res. 55 (4), 485–540. https://doi.org/10.3102/
00346543055004485.
Berge, Z. L., & Huang, Y. P. (2004). Model for sustainable student retention: a holistic
perspective on the student dropout problem with special attention to e-learning. htt
ps://pdfs.semanticscholar.org/ee12/583d7489e6da6e293a00489a5df39e149454.
pdf?_ga=2.62704681.402980605.1593554259-274620667.1593554259.
Billings, D.M., Faan, E.R., Halstead, J.A., 2019. Teaching in Nursing: A Guide for Faculty
(Elsevier Health Sciences).
Black, E.W., Dawson, K., Priem, J., 2008. Data for free: using LMS activity logs to measure
community in online courses. Internet High. Educ. 11 (2), 65–70. https://
doi.org/10.1016/j.iheduc.2008.03.002.
Cameron, J., Roxburgh, M., Taylor, J., Lauder, W., 2011. An integrative literature review
of student retention in programmes of nursing and midwifery education: why do
students stay? J. Clin. Nurs. 20 (9–10), 1372–1382. https://doi.org/10.1111/j.1365-
2702.2010.03336.x.
Canadian Nurses Association, 2008. Registered Nurses Education in Canada Statistics
2006–2007. Canadian Nurses Association, Ottawa, Canada.
Canadian Nurses Association, 2019. Registered Nurses Education in Canada Statistics
2017–2018. Canadian Nurses Association, Ottawa, Canada. https://www.casn.ca/w
p-content/uploads/2019/12/2017-2018-EN-SFS-DRAFT-REPORT-for-web.pdf.
Fredricks, J.A., McColskey, W., 2012. The measurement of student engagement: A
comparative analysis of various methods and student self-report instruments. In:
Christenson, S.L., Reschly, A.L., Wylie, C. (Eds.), Handbook of Research on Student
Engagement. Springer, Boston, MA, pp. 763–782.
Pittenger, A.L., Lounsbery, J.L., 2011. Student-generated questions to assess learning in
an online orientation to pharmacy course. Am. J. Pharm. Educ. 75 (5), 94. https://
doi.org/10.5688/ajpe75594.
Polit, D.F., Beck, C.T., 2008. Nursing Research: Generating and Assessing Evidence for
Nursing Practice. Lippincott Williams & Wilkins.
Richardson, J.T., 2011. Eta squared and partial eta squared as measures of effect size in
educational research. Educational Research Review 6 (2), 135–147. https://doi.org/
10.1016/j.edurev.2010.12.001.
Rovai, A.P., 2004. A constructivist approach to online college learning. Internet High.
Educ. 7 (2), 79–93. https://doi.org/10.1016/j.iheduc.2003.10.002.
Tinto, V., 1998. Colleges as communities: taking research on student persistence
seriously. Rev. High. Educ. 21 (2), 167–177.
You, J.W., 2016. Identifying significant indicators using LMS data to predict
course achievement in online learning. Internet High. Educ. 29, 23–30.
https://doi.org/
REFERENCES AND CITATIONS:

Article #1:

Linda Cole, Lisa Boss, Debra Fowler, Latarsha Cheatham,

Using Leadership Simulation Scenarios With Graduate Nursing Learners to Support Frontline Nurse
Leader Competency Development,
Nurse Leader,
Volume 19, Issue 4,
2021,
Pages 405-410,
ISSN 1541-4612,
https://doi.org/10.1016/j.mnl.2020.08.012.
(https://www.sciencedirect.com/science/article/pii/S1541461220302226)
Abstract: Although the use of simulation has grown in education, there is little written about how it
can be used with frontline nurse leaders (FLNLs) to teach nontechnical skills such as critical thinking,
teamwork, communication, and conflict resolution. This article discusses the implementation of FLNL
simulation in a graduate nurse leadership program based on the American Organization for Nursing
Leadership (AONL) Nurse Manager Competencies. Learner confidence increased in the AONL
competencies post-simulation. Although the setting of this project was the academic setting, the
approach described could be easily adapted to the clinical setting for orientation and education of
FLNLs.

Article # 2
Kate Jones, Liam C. Hein, Lisa James,
A Nursing Leadership Practicum in the Time of COVID19:: A Southeastern University Experience,
Nurse Leader,
Volume 19, Issue 2,
2021,
Pages 145-149,
ISSN 1541-4612,
https://doi.org/10.1016/j.mnl.2020.06.010.
(https://www.sciencedirect.com/science/article/pii/S1541461220301737)
Abstract: Coronavirus disease 2019 (COVID19) shutdowns have impacted nursing education,
particularly student clinical experiences. In this paper, we discuss how we adapted an accelerated
MSN nursing administration clinical course to meet the needs of our practicing students and the
Commission on Collegiate Nursing Education (CCNE) standards in the context of the COVID19
pandemic. Resources available through the American Organization for Nursing Leadership (AONL)
were utilized. Students reported a supportive, valuable learning experience they could apply in their
future practice as nurse leaders. Considerations for future revisions to the course and to the AONL
Guiding Principles are also presented.
Article # 3:

M.A. De Juan Pardo, P. Fuster, A. Gallart, E. Rodríguez, L. Wennberg, M.L. Martin-Ferreres,

Fostering leadership competence and satisfaction in nursing undergraduates through a student-led
conference: A quasi-experimental pre-post study,
Nurse Education Today,
Volume 98,
2021,
104748,
ISSN 0260-6917,
https://doi.org/10.1016/j.nedt.2021.104748.
(https://www.sciencedirect.com/science/article/pii/S0260691721000058)
Abstract: Background
Numerous benefits have been reported for student-led conferences, such as increased leadership.
This competence has been recognized as important for nurses so as to ensure the provision of safe
and high-quality care in complex environments. However, research has yet to examine empirically the
impact of student-led conferences on students' leadership behaviours.
Objectives
To examine the impact that participation in a student-led conference had on the self-perceived
leadership competence of nursing undergraduates.
Design
Quasi-experimental single group pre-post intervention study.
Setting
Faculty of Medicine and Health Sciences at the Universitat Internacional de Catalunya.
Participants
31 students enrolled in two elective modules offered during the final year (fourth year) of a nursing
degree programme.
Methods
Pre-post assessment of self-perceived leadership behaviours among nursing students involved in
planning and organizing a scientific conference. In addition to carrying out the tasks of organizing the
Conference, all students participated as co-authors of an oral communication, thus being able to
develop both cognitive and non-cognitive domains. Leadership was measured using ES_SALI scale, the
Spanish version of the Self-Assessment Leadership Instrument.
Results
Involvement in the student-led conference led to a statistically significant increase in self-perceived
leadership competence among nursing undergraduates (p < .001). Both the total ES_SALI score and
scores on each of its four dimensions (Strategic thinking, Emotional intelligence, Impact and influence,
and Teamwork skills) increased significantly, and the percentage change was above 8% in all cases
(p < .01). The greatest increase (10.99%) corresponded to the ‘Impact and influence’ dimension of
leadership.
Conclusions
The results suggest that student-led conferences are an effective way of helping nursing
undergraduates to develop their leadership competence.
Keywords: Communication skills; Critical thinking; Emotional intelligence; Leadership; Nursing
students; Student-led conferences; Teamwork

Article # 4:
Gözde Özaras Öz, Süheyla Abaan,
Use of a flipped classroom “Leadership in Nursing” course on nursing students' achievement and
experiences: A quasi-experimental study,
Journal of Professional Nursing,
Volume 37, Issue 3,
2021,
Pages 562-571,
ISSN 8755-7223,
https://doi.org/10.1016/j.profnurs.2021.02.001.
(https://www.sciencedirect.com/science/article/pii/S8755722321000247)
Abstract: Background
Providing the undergraduate “Leadership in Nursing” course using an innovative approach is
regarded to be very important in terms of enhancing leadership skills.
Purpose
The purpose of this study was to evaluate the effect of using the “flipped classroom” strategy on
nursing students' learning achievements and to describe their experiences of this technique.
Method
The course of “Leadership in Nursing” was delivered in flipped classroom format for 20 students while
another 19 were exposed to traditional teaching methods. Therefore, the study was completed with
39 students. Data was collected using “Student Introductory Information Form”, “Exams (a midterm
exam, a final exam and 4 assignments)”, “In-Class Observation Form” and “Student Feedback Form”.
Findings
Students in the flipped classroom group had obtained significantly higher scores than the students in
the traditional teaching group regarding the assignments, the final exam, and overall grade.
Concerning the flipped classroom, students reported flexibility as the most positive aspect and
problems related to the system infrastructure as the most negative aspect.
Conclusion
The usage of a flipped classroom for nursing students in the leadership course provided valuable
results in terms of the students' exam scores and achieving course objectives. The flipped classroom
strategy for the “Leadership in Nursing” course was shown to be effective in improving abilities such
as distinguishing leadership skills, visioning, developing a new perspective, willingness to share
experiences regarding leadership, also enabling students to work in harmony in group activities and to
participate in classroom discussions, which are all essential elements of leadership.
Keywords: Nursing; Leadership; Flipped classroom; Traditional teaching; Learning outcomes
Article # 5
Samantha Juan,
Promoting engagement of nursing students in online learning: Use of the student-generated
question in a nursing leadership course,
Nurse Education Today,
Volume 97,
2021,
104710,
ISSN 0260-6917,
https://doi.org/10.1016/j.nedt.2020.104710.
(https://www.sciencedirect.com/science/article/pii/S0260691720315604)
Abstract: Background
The frequency of online learning has surged in nursing education, and engagement is a critical
element in students' success in online learning. An evidence-based learning tool is needed to promote
engagement in online nursing education.
Objectives
To examine the effectiveness of the student-generated question (SGQ) in promoting engagement in
an online nursing leadership course.
Design
A quasi-experimental and post-test design.
Setting
The online nursing leadership course at a university in Canada.
Participants
179 undergraduate nursing students.
Methods
Six learning behaviors related to engagement, including number of visits, time spent, topics visited,
posts read, threads created, and replies posted, were retrieved from the institution's learning
management system (LMS). The equality of covariance matrices was applied to examine the
homogeneity of the six outcome variables among the control and intervention groups, and
multivariate analysis of variance (MANOVA) was conducted to compare the statistical differences in
all six outcome variables for three planned phases of instruction.
Results
The equality of the covariance matrices indicated that the six outcome variables were homogeneous
across the control and intervention groups. The multivariate test showed a statistically significant
difference between the control and intervention groups in the post-test measurements. In addition,
the MANOVA results demonstrated statistically significant differences in the two phases when SGQ
was implemented but was statistically insignificant when SGQ was not implemented. The intervention
group demonstrated higher means in the two phases in which SGQ was implemented.
Conclusions
The findings support SGQ as an effective learning tool for promoting engagement in an online nursing
leadership course.
Keywords: Nursing students; Engagement; Online learning; Student-generated question (SGQ);
Leadership course