| |

Received: 6 September 2019    Revised: 23 October 2019    Accepted: 5 November 2019

DOI: 10.1111/jan.14270

PROTOCOL

A mixed methods study to evaluate the effects of a teamwork

enhancement and quality improvement initiative on nurses’
work environment

Jenniffer Torralba Paguio MA (Nursing), RN, PhD Candidate, Assistant Professor1,2  |

Doris Sau Fung Yu PhD (Nursing), BSc (Nursing Studies), Professor1

1
Nethersole School of Nursing, Faculty of
Medicine, The Chinese University of Hong
Kong, Shatin, Hong Kong
2
College of Nursing, University of the
Philippines–Manila, Manila, Philippines
Correspondence
Jenniffer Torralba Paguio, Nethersole
School of Nursing, Faculty of Medicine, The
Chinese University of Hong Kong, Shatin,
Hong Kong.
Email: jtpaguio@link.cuhk.edu.hk
Funding information
The Association of Nursing Service
Administrators of the Philippines (ANSAP),
Inc., funded this study.
Abstract
Aims: To test the effect of a teamwork enhancement and quality improvement pro-
gram named ‘Nurturing Effective Teams and Continuous Quality Improvement’ on
nurses’ work environments in the hospital setting.
Design: Mixed-method study with quasi-experimental controlled trial and focus
group interviews.
Methods: Twelve units from two Philippine tertiary government hospitals will be
matched and allocated to have 72 nurses receiving the teamwork enhancement and
quality improvement program, or no intervention. The program focuses on enhanc-
ing the nurse autonomy, leadership and management support, teamwork, and work-
load management, delivered in two phases: (a) teamwork enhancement training using
team strategies and tools to enhance the performance and patient safety; and (b)
implementation of quality improvement projects using the model for improvement
to identify priority unit issues and change ideas and tested using the Plan-Do-Study-
Act cycle. The 6-month implementation includes two trainings and seven mentoring
sessions with a quality improvement facilitator. Primary outcome is the nurses’ work
environment and secondary outcomes are job satisfaction, burnout risk, turnover
intention, and perceived quality of care provided measured at the nurse-level using
self-administered survey and measured at 0, 3, and 6 months. Focus group interviews
will be conducted among 14–16 nurse subjects to explore their experience during the
program, while other stakeholders will be interviewed to reflect the program effects.
Generalized equation modelling will be used to identify the program effects on the
quantitative outcomes and content analysis will be used for qualitative data.
Discussion: Establishing measures to improve the nurses’ work environments can be
used to address poor nurse outcomes in high workload and low-resource settings.
Impact: A favourable nurse work environment is the cornerstone to sustainable nurs-
ing workforce and positive outcomes. This study will provide explicit evidence to
inform the effect of a structured evidence-based protocol in improving nurses’ work
environment given resource-limited context.

The peer review history for this article is available at https ://publo ns.com/publo
n/10.1111/jan.14270

|
664     © 2019 John Wiley & Sons Ltd wileyonlinelibrary.com/journal/jan J Adv Nurs. 2020;76:664–675.
PAGUIO et al.
Trial Registration: China Clinical Trial Registration Center (CciCTR), Registration num-
ber ChiCTR1900021754 (March 8, 2019).
KEYWORDS
autonomy, burnout, job satisfaction, mixed method, nurses, nursing, quality improvement,
teamwork, turnover, work environment
1 |  I NTRO D U C TI O N
High-quality hospital service plays a key role in addressing the global
agenda of optimizing population health (World Health Organization,
2016). As such, healthcare organizations and health policy makers
place great emphasis on identifying conditions with the highest
impact on achieving better health outcomes. As a major factor in
health care, significant efforts have explored the role of the work en-
vironment of nurses in enhancing the quality of healthcare delivery
(International Council of Nurses, 2018). Nurses’ work environment
(NWE), which describes the context and physical environment for
nurses to deliver health care, has emerged as a terminology to sup-
port this investigation.
Nurses’ work environment is most widely defined as “a profes-
sional practice environment [that] supports nurses to function at the
highest scope of clinical practice, to work effectively in an interdisci-
plinary team of caregivers and to mobilize resources quickly.” (Lake,
2007, p. 106S). This definition provides important insight on what
characterizes an effective nursing practice environment. It also hints
that NWE is a highly context-specific concept that is shaped by the
nature of the healthcare system and its policies, while its improve-
ment is challenged by health human resources availability, patient
care demand and the health sector.
There is an urgent need to address the poor global nursing work
conditions describing NWE confronted by health human workforce
shortage, migration, low professional nurse status, unjust work com-
pensation, and lack of nursing positions, commonly encountered
in developing countries and low-resource settings (International
Council of Nurses, 2018). Despite this, reviews continue to demon-
strate a lack of high-quality research evidence to inform strategies
to improve work environment of nurses and enhance the quality of
health care (Schalk, Bijl, Halfens, Hollands, & Cummings, 2010; Wei,
Sewell, Woody, & Rose, 2018).
1.1 | Background
The Donabedian structure-process-outcome model provides the
framework to inform how to improve NWE for better healthcare
outcomes. (Donabedian, 1988, 2005). Structures refer to the or-
ganizational characteristics such as manpower structure, scope
of service and policy, which affect how the organization provides
health care (Stalpers, de Brouwer, Kaljouw, & Schuurmans, 2015).
Process characteristics refer to all the conditions, which directly
|
      665
affect the nurses' ability to provide care such as the presence of
nurse manager support and leadership, autonomy and interac-
tion with healthcare team members and care providers. Finally,
outcomes are the results of the interaction of structures and pro-
cesses and can be measured in terms of nurse-sensitive patient
outcomes, nurse outcomes, and hospital outcomes. Relationships
between structures, processes and outcomes, provide a good indi-
cator of the state of the NWE and allows for testing innovation for
improvement. Figure 1 illustrates these relationships. According to
this model, intervention to improve NWE needs to focus on shap-
ing the structure and process characteristics in a way to benefit the
healthcare delivery.
A substantial body of evidence supports positive organizational
structures characterized by adequate workforce of competent
nurses, and encouraging nurses’ autonomy and enabling supportive
management on patient-, nurse- and hospital outcomes would have
positive impact on health and health service outcomes. These NWE
have been associated with positive patient outcomes of shorter hos-
pitalization, lower missed care, reduction in patient safety incidents
(i.e., falls, errors, infections), lower mortality, better patient satisfac-
tion, nurse retention and satisfaction, and reduction of job stress
and hospital costs and safe hospital culture (Aiken et al., 2016; Yoo
& Kim, 2017; MacPhee, Dahinten, & Havaei, 2017; McHugh & Ma,
2014; Twigg & McCullough, 2014).
Nevertheless, with the continued challenge to improve NWE,
research evidence to inform effective ways to improve NWE is very
limited. One such review done in the past decade identified four
different strategies, including primary nursing, an educational tool-
box, clinical supervision and violence prevention, to improve NWE,
but the effectiveness of these strategies cannot be determined due
to methodological issues (Schalk et al., 2010). This is affirmed by a
more recent review by Wei et al. (2018) that most studies on NWE
are at a descriptive level and stringent evaluation of their effects
were hindered by the lack of use of experimental study design.
Moreover, the generalizability of such evidence is also confined to
the context of countries where there are adequate healthcare re-
sources (Australia, Belgium, Canada, Germany, S. Korea, UK, USA).
As such, the ICN has set out a highly prioritized agenda to improve
the NWE through more vigorous research from all contexts and set-
tings (2018). In response to ICN's advocacy, this protocol uses the
Donabedian model to provide the theoretical basis to inform the
development of a NWE improvement program and its effects in a
healthcare setting of low resources and high workload demand were
evaluated.
 |
666       PAGUIO et al.

F I G U R E 1   Relationship of structure
and process variables of the nurses’ work
STRUCTURES PROCESSES
environment (NWE) with Outcomes
[Colour figure can be viewed at
wileyonlinelibrary.com]
HOSPITAL CHARACTERISTICS
PRESENCE OF MANAGERIAL
Bed Capacity
SUPPORT & LEADERSHIP
Hospital status

NURSE CHARACTERISTICS
Employment status
Years experience NURSE AUTONOMY
Adequacy of Salary
Engagement in continuing
profesional development

NURSE-PHYSICIAN
PATIENT CHARACTERISTICS RELATIONS
Acuity

NURSE WORK ENVIRONMENT (NWE)

OUTCOMES
Nurse
Patient
Hospital

2 |  TH E S T U DY
2.1 | Aims
The aims of this mixed-method study were of twofold. First, it will
examine the effects of Nurturing Effective Teams and Continuous
Quality Improvement (NET-CQI) program on outcome (NWE, job
satisfaction, burnout, intention to leave and perceived quality of
care). Second, the study also aims to explore the nurses’ percep-
tions about the NWE improvement program and their experiences
throughout the intervention period and describe the impact of the
NET-CQI program, on other healthcare workers, administrators, and
patients.
2.2 | Research hypotheses
The hypotheses include:
1. Nurses who participated in the NET-CQI program will have
a greater improvement in job satisfaction, burnout risk and
intention to leave than those in the comparison group;
2. Nurses who participated in the NET-CQI program will have a bet-
ter perception of the NWE in terms of nurse autonomy, received
leadership and management support, and relationship with physi-
cians as measured by the PES-NWI; and
3. Nurses in units who implemented the NET-CQI program will per-
ceive better quality of care provided compared with units without
the program
2.3 | Design and methodology
This is a mixed-method, non-equivalent group, repeated measures
quasi-experimental study, with focus group interview, conducted in
two tertiary government hospitals in the Philippines. Two institu-
tions from the preliminary study phase with comparable baseline
measures (nursing workforce and workload, patient and hospital
profile) will be selected. These hospitals will be randomly assigned
as the study setting for implementing the program or control re-
spectively. Twelve units from each hospital will be matched based
on their service focus (e.g., surgery, medicine, paediatrics), service
level (general, critical care), authorized bed capacity, and nurse
position.
PAGUIO et al.
2.4 | Participants
Subjects will be eligible if they are full-time employed nurses in units
providing direct patient care. No previous similar studies focusing on
impact of work environment interventions reported effect sizes. As
the CQI-NET is a complex intervention, a priori medium effect size of
0.5 was therefore set to indicate the effectiveness of the interven-
tion on the outcomes. Using the power analysis software Gpower
3.1, it is estimated that a sample size of 64 participants per arm would
give the two-arm trial 80% power to detect an effect size of at least
0.5 at two-sided 5% level of significance. Allowing for up to 10% at-
trition rate, it is targeted to recruit at least 72 participants per arm.
To achieve this target sample, an average of 6–8 nursing staff from
each of the 10–12 matched units from the hospitals will be recruited.
Participants for focus group interviews will include nurses who
participated in NET-CQI and other healthcare personnel and pa-
tients from the intervention units. Six groups of nurses (5–8 par-
ticipants per group), with two in each of the categories of (a) nurse
administrators, (b) staff nurses in permanent positions, and (c) con-
tractual nurses, will be recruited. One group of healthcare personnel
(including physicians and unit administrators) and another group of
patients who were in the unit during the implementation process
period of the NET-CQI will be recruited to capture their perceptions
about the quality of healthcare delivery. The other eligibility criteria
for the patients include at least 21  years old, admitted in the unit
for at least 5 days during the intervention period, able and willing to
participate in the FGD. The healthcare personnel and patients will be
recruited from different units for better representation.
2.5 | Study Intervention
Nurturing Effective Teams and Continuous Quality Improvement pro-
gram is a process improvement initiative to improve NWE in hospital
unit levels. As ‘NET’ also symbolizes a mesh to bring together varied
aspects of the work environment, NET-CQI aims to foster team cohe-
sion through quality improvement activities. The two-phase program
focuses on the following areas: (a) enhanced autonomy in role and
practice; (b) enhanced leadership and management skills in providing
support; (c) improved communication; and (d) workload management.
Phase 1 is essential to the implementation of Phase 2. The purpose
of the first phase is to empower the nurses with the understanding
of the NWE concept, its impact on nurse-, patient-, and organization
outcomes and the integral role of teamwork in promoting favourable
NWE. Using the concepts learned from Phase 1, the nurses will deter-
mine their focus for QI projects and implement these in Phase 2. The
overall program will be implemented for 6 months and in two phases.
2.5.1 | Phase 1 (week 1–week 2)
This phase is composed of two sessions: (a) discussion on NWE; and
(b) the teamwork enhancement training. The discussion on NWE
|
      667
focus on understanding of the structures and processes that influ-
ence the NWE and the important role of nurses in ensuring healthy
organizational processes, through a 1-hour activity. The PI will lead
the lecture, open forum, and small group discussions. The PI will
highlight the findings from the NWE assessment of the institution
and the insights generated from the national state of government
hospitals to (a) establish a shared mental model essential for team-
work; and (b) identify problems, issues, and priorities of nurses at
the hospital and units levels to inform their quality improvement
projects. The PI will also discuss evidence-based recommendations
on how to achieve healthy NWE and the positive influence of team
strategies on healthcare team members and patients.
The 6-hour teamwork enhancement training workshop fol-
lows the Agency and Healthcare Research and Quality's (Agency
for Healthcare Research & Quality, 2019) TeamSTEPPS® 2.0
Foundations course aimed at empowering nurses’ team perfor-
mance conducted by a TeamSTEPPS® Master Trainer and assisted by
TeamSTEPPS®-trained nurses. Training activities will follow the pre-
scription of the standard TeamSTEPPS® 2.0 Foundations modules.
Each module takes 30–60 min to deliver that include short lectures,
small group activities, video reaction exercises, and use of work-
sheets (Table S1). This crucial training will enable the unit teams to
implement their quality improvement projects effectively in Phase
2. The eight modules of the teamwork enhancement training include
‘Introduction to TeamSTEPPS’, ‘Team Structure’, ‘Communication’,
‘Leading Teams’, ‘Situation Monitoring’, ‘Mutual Support’, ‘Putting it
all Together’, and ‘Change Management’. The ‘Change Management’
module is an additional module from the TeamSTEPPS® Master
Trainer course (AHRQ, 2019) intended to help nurses implement
and train others in TeamSTEPPS® is included and will focus on the
concept of implementing and managing change interventions in the
organization. All the activities of Phase 1 will be delivered in 1 day
but will be offered three times for the duration of 2 weeks to accom-
modate the schedule of nurses and ensure maximum participation.
Close coordination with the hospital administration will be done to
facilitate scheduling and availability of the participants. By the end
of the Phase 1, participants will be asked to reflect on their units’
potential areas of improvement in preparation for Phase 2 activities.
2.5.2 | Phase 2 (week 2–week 24)
A QI mentor who has completed the IHI modules on quality improvement
and experience in conducting research will lead the training on quality
improvement based on the model for improvement and the implemen-
tation of unit-based projects ‘Plan-Do-Study-Act’ (PDSA) cycle. Phase 2
has two parts: (a) a quality improvement workshop; and (b) a quality im-
provement implementation. One QI mentor will conduct the training and
guide the participants throughout the implementation period.
Quality improvement workshop
This 8-hour workshop will be delivered through short lectures and
small group activities, focused on building the nurses’ competencies
 |
668       PAGUIO et al.
in quality improvement using AHRQ's model for improvement
(Figure S1). This model is composed of two parts: (a) identification
of the change for improvement using a three-question approach
including ‘What are we trying to accomplish?’, ‘How will we know
a change is an improvement?’, and ‘What change can we make that
will result in improvement?’; and (b) the PDSA cycle. The training will
use the IHI Quality Improvement Essentials Toolkit (IHI, 2018) that
includes ‘Cause and Effect Diagram’, ‘Driver Diagram’, ‘Flowchart’,
‘Data Presentation’, ‘QI Project Charter’, ‘Project Planning Form’,
and ‘PDSA Worksheet’ (Table S2). This will conclude with a pres-
entation of the NET-CQI intervention packages (Figure 2) that will
guide unit teams in selecting their QI projects. Like the Phase 1
training, this workshop will be offered three times for 2 weeks.
Quality Improvement Implementation
After equipping the nurses on quality improvement competencies, each
unit QI team will run their projects guided by a QI mentor through three
face-to-face and four web-based critical pathway-coaching sessions
spread throughout 22 wk, to change the NWE at the unit level. These
critical pathway-coaching sessions address four key domains of effec-
tive NWE including promotion of nursing autonomy, enhancement of
leadership and management skills, improving communication skills, and
effective workload management (Figure 2). These four key areas were
identified from a preliminary study by the research team as the most
prevalent shortcomings of the NWE of the study settings (Paguio & Yu,
2019). Each domain encompasses two potential QI projects based on
effective strategies outlined in literature to improve the corresponding
dimension of NWE (Aiken et al., 2018; Alonso et al., 2006; Ceravolo et
al., 2012; Gittell, Beswick, Goldmann, & Wallack, 2005; Rao, Kumar, &
McHugh, 2017; Weston, 2010). Table 1 presents the projects in each
area and the associated working pathways. Regardless of the focus of
the QI project, the QI mentor will guide the unit teams following the
model for improvement's three questions and PDSA cycle through a
three-step approach to support the nurses to change the NWE:
1. Identifying the core issue and strategies for NWE improvement
(Weeks 3 to 5). The QI mentor will guide the nurses to focus on
their unit's key issues identified during the Phase 1 workshops.
Then, the PDSA cycle will be applied to address the issue.
Using the ‘Cause and Effect Diagram’, ‘Driver Diagram’, and the
‘Process Flow’ worksheets, QI teams will define their specific
target area for improvement. Once finalized, the three-question
approach will be used to identify the change needed for NWE
improvement and the ‘Project Charter’ will be used to facilitate
the process. Each unit will address the question ‘What are we
trying to accomplish?’ by exploring their unit issues and agreeing
on the priority unit problem, its description, rationale, expected
benefits and outcomes, and coming up with their project aim
statement. The units will then determine their project measures
to answer the question ‘How will we know that a change is
an improvement?’. To answer the question ‘What changes can
we make that will result in improvement?’, the QI mentor will
help the unit identify their initial activities, change ideas, key
stakeholders, barriers, and boundaries in the implementation
phase and will discuss the possible resolution. Modifications
in the QI plan will be made accordingly during the first face-
to-face and subsequent first web-based mentoring sessions.
2. Planning the QI project (Weeks 5–9). This step involves develop-
ing the project plan using the ‘Project Planning Form’ and ‘PDSA
Worksheet’. The unit teams will systematically plan their projects
using the ‘Project Planning Form’ by reflecting their project driv-
ers, measures (outcome, process, and balancing), goals or targets,
change ideas, list of tasks to prepare for their project testing, per-
sons responsible, and timeline. They will work out the method of
evaluating the change resulted from implementing the action plan
using the ‘PDSA Worksheet’. The QI mentor will guide the teams in
running their first PDSA cycle focusing on testing the feasibility of
their set measures (outcome, process, balancing). Outcome meas-
ures are those that quantify the goal attainment (e.g., Reduce infec-
tion rate by 50%). Process measures are those that correspond to
each change idea (e.g., Percentage of nurses complying with hand-
washing protocol), while balancing measures are those that indicate
whether their test of change is introducing problems in the system
(e.g., Increase in expenses of handwashing supplies). The PDSA
cycle will involve defining their measures (Plan), taking their baseline
measures (Do), analyse (Study) and decide to modify or retain their
measures and start their change idea testing (Act). This step will be
carried out from weeks 5–9 and involve the second face-to-face
session and the subsequent second web-based mentoring sessions.
3. Implementing the QI project (Weeks 10 to 24). The unit teams
will implement the QI plan and will be guided by the QI mentor
through one face-to-face and two web-based mentoring sessions
throughout the project implementation. Unit teams will docu-
ment their tests of change using the ‘PDSA Worksheet’ and will
run several PDSA cycles depending on their project measures.
The teams will use the ‘Data Presentation’ tools to organize their
project results. Once the PDSA cycles are completed, they will
analyse and present their final project reports.
2.6 | Data collection
Data collection will take place after obtaining approval from the hospital
ethics committees. This study will collect both quantitative and qualita-
tive data. While the quantitative data collection focuses on outcome
evaluation for CQI-NET, the qualitative data aim to capture nurses’ ex-
periences with the tested intervention. Figure 3 illustrates the flow of
data collection and implementation of the 6-month intervention.
2.6.1 | Quantitative outcome evaluation
For the outcome evaluation, the following will be collected at base-
line (T0), at 3 months (T1) and at 6 months (T2) of the NET-CQI pro-
gram thereafter for all the participants.
PAGUIO et al. |
      669

Priority Unit Issue QI Project Package NWE Process Domain

Lack of specific Test the effectiveness of a

A-1
nursing skill skills-enhancement activity

Autonomy

Test the effectiveness of a

Ineffective nursing
nurse-developed /adapted A-2
care practice
modification in patient care

Test the effectiveness of

tools to provide mutual B-1
support (task assistance,
Lack of teamwork in providing feedback)
Leadership and
an inexperienced
Management
team or high demand
support
(busy) units Test the effectiveness of
tools to provide lead care
teams (briefs, huddles, B-2
debriefs)

Test the effectiveness of

tools to enhance
information exchange C-1
(SBAR, call-outs, check-
Poor communication backs) Relationship
between nurses and among nurses,
other healthcare team and between RN-
members MD
Test the effectiveness of
techniques to advocate for
patients during referrals
and management conflict C-2
(CUS, two-challenge rule,
DESC script)

Test the effectiveness of a

new scheduling scheme D-1

Workload
Heavy workload
management
Test the effectiveness of a
new strategy to allocate D-2
workload

F I G U R E 2   Intervention packages for Nurturing Effective Teams and Continuous Quality Improvement (NET-CQI) program [Colour figure
can be viewed at wileyonlinelibrary.com]
 |
670       PAGUIO et al.

TA B L E 1   NET-CQI proposed quality improvement packages indicate higher emotional exhaustion similar to the approach of
based on NWE domains Aiken, Buchan, Ball, and Rafferty (2008). The Cronbach's alpha of
NWE Domain Package Description MBI-HSS is 0.88. Factor analysis indicates its good construct validity
in measuring burnout. (Aiken et al., 2008).
Promoting A1 Testing the effectiveness of a skills-
nurse enhancement activity
autonomy A2 Testing the effectiveness of a nurse- Intention to leave and quality of care provided
developed or adapted modification in As no standardized instruments are available to measure intention to
patient care processes leave and perceived quality of care, the measuring methods used in previ-
Enhancing B1 Testing the effectiveness of tools to ous studies on NWE will be used (Aiken et al., 2008; Van Bogaert et al.,
leadership provide mutual support (task assistance, 2017). Three dichotomous questions will be used to determine the nurses’
and manage- providing feedback)
intention to leave the unit, hospital, and country) in the next 6 months to
ment skills B2 Testing the effectiveness of tools to lead
1 year. On the other hand, a ‘1-4’ Likert scale will be used to determine
care teams (briefs, huddles, debriefs)
nurses’ perceived quality of care in the unit last year and previous shift,
Improving C1 Testing the effectiveness of tools to en-
and in the hospital, where higher scores indicate better quality of care.
communica- hance the information exchange (SBAR,
tion skills call-outs, check-backs)
C2 Testing the effectiveness of techniques
to advocate for patients during referrals 2.6.2 | Qualitative data collection
and management conflict (CUS, two-
challenge rule, DESC script)
A 1-hour focus group interview will explore the experiences of nurses
Promoting D1 Testing the effectiveness of a new throughout the intervention period. Interview questions are presented
effective scheduling scheme
on Table 2. An independent data collector who did not take part in any
workload D2 Testing the effectiveness of a new strat-
management of the intervention activities will moderate the focus group interview
egy to allocate workload
and a research assistant will be an observer and take field notes. Field
Abbreviation: NET-CQI, Nurturing Effective Teams and Continuous notes and audio recordings will be kept, to ensure accuracy.
Quality Improvement; NWE, nurses’ work environment.

Nurse work environment 2.6.3 | Collection of contextual variables

The 31-item Practice Environment Scale of the Nursing Work Index (PES-
NWI) will measure the overall NWE and its dimensions: ‘Participation in
Hospital Affairs’ (9 items) and ‘Nursing Foundations for Quality of Care’
(10 items), ‘Nurse Manager Ability, Leadership and Support’ (5 items),
‘Staffing and Resource Adequacy’ (4 items), and ‘Collegial nurse–phy-
sician relationship’ (3 items). A ‘1-4’ Likert scale is used, where higher
mean scores indicate more favourable NWE. The PES-NWI has good
internal consistency (Cronbach's alpha = 0.71–0.84), inter-item correla-
tion (0.64–0.91), and reliability (ICC  =  0.88–0.97; Lake, 2014, 2002).
The good factorial validity also supports the use of both composite
score and five dimensions scores (Ogata et al., 2018).
Job satisfaction
This is measured using the job satisfaction dimension (5 items) of the
Safety Attitudes Questionnaire-Short Form (SAQ-SF) based on a ‘1-5’
Likert scale, with a composite score of ≥75% indicating higher nurses’
positivity about work experience. The SAQ-SF has good internal con-
sistency (Cronbach's alpha  =  0.91), acceptable inter-item correlation
(0.47–0.58), and good reproducibility (IDC 0.81–0.92; Sexton et al.,
2006). The good factorial validity also supports the use of the dimen-
sion score (Jeon, Park, Choi, & Kim, 2018; Sexton et al., 2006).
Contextual variables to describe the sample will be collected including
information about the nurses, workplace characteristics, and patient
acuity. The nurse characteristics will include nurses’ employment
(contractual or permanent), current salary, numbers of attending pro-
fessional training in the past year, years of experience, and years work-
ing in the current hospital. The workplace characteristics will include
mandated duty hours, time spent for direct care, number of assigned
patients based on previous shift, number of nurses involved in pro-
viding direct care, and nurse-rated patient acuity. Patient acuity will
be measured using the 8-item Patient Acuity Rating proposed by the
American Association of Critical-Care Nurses (AACN) Synergy Model
used to measure patient resiliency, vulnerability, stability, complexity,
resource availability, participation in care, participation in decision-
making, predictability. A ‘1-5’ Likert scale is used with higher scores
representing higher susceptibility. The instrument has good internal
consistency (Cronbach's alpha = 0.88) and discriminant construct va-
lidity between novice and expert nurses (Brewer et al., 2007).
2.7 | Validity and reliability

Burnout risk 2.7.1 | Data collection method

The five-item Emotional Exhaustion subscale of Maslach's Burnout
Inventory-Human Services Survey (MBI-HSS) will be used to meas- Nurses in the Philippines are fluent in spoken and written English,
ure the burnout risk using a ‘1-7’ Likert scale, where higher scores after completing a 4-year baccalaureate degree, the national
PAGUIO et al. |
      671

Recruitment of study units and nurses Recruitment of matching units

Baseline assessment (T1) Baseline assessment (T1)

Intervention Units Hospital Matched Control Units Hospital

72 nurses 72 nurses

Phase I: NWE Orientation

Phase II: Teamwork Enhancement
Training: 8 hours

Phase III: PDSA-guided Quality Improvement (QI) cycle

Training: 8 hours
Milestone Activities: Group presentation, Mentoring Meetings
(Face-to-face, Web-based)

Outcome Evaluation (T2) Outcome Evaluation (T2)

Phase III: PDSA-guided Quality Improvement (QI) cycle

continuation
Milestone Activities: Group presentation, Mentoring Meetings
(Face-to-face, Web-based)

Process Evaluation

Outcome Evaluation (T3) Outcome Evaluation (T3)

Focus Group Discussions

Data Encoding & Entry

Transcription

Quantitative Data Analysis

Qualitative Data Analysis

Synthesis of Study Findings

F I G U R E 3   Data collection plan [Colour figure can be viewed at wileyonlinelibrary.com]

 |
672       PAGUIO et al.
TA B L E 2   Focus group discussion questions
Participant
category Questions
Nurses 1. Describe your experiences throughout the
trainings.
2. Describe your experiences throughout your pro-
ject implementation.
3. What do you think are the barriers and facilitators
to your project?
4. What have you learned about yourself and your
team throughout this process?
5. How do you think your QI project influences your
unit's work environment?
6. What do you think about the last 6-month's activi-
ties and any changes in you and your unit?
Doctors, 1. What have you observed among the nurses in this
non-nurse unit over the last 5 to 6 mo?
adminis- 2. In what way do think their unit project [insert unit
trators project] has affected the unit's personnel, work
environment, and interaction with you?
3. How do you think projects like these affect the
nurses, healthcare personnel, patients and their
carers, and this hospital?
Patients 1. What have you observed among the nurses in this
and care unit during your hospital stay?
providers 2. In what way do you think their unit project [insert
unit project] has affected the unit's personnel,
work environment, and interaction with you?
3. How do you think projects like these affect
patients and their carers, the nurses and other
healthcare personnel and this hospital?
professional examination and using the language in all hospital docu-
mentation. As such, translation of the survey instrument to Filipino
was not done (Turnipseed & Turnipseed, 1997) and has been suc-
cessfully used in a preliminary phase after approval from a national
ethics review board (SJREB-2018-05).
All the study instruments have acceptable reliability and valid-
ity. In addition, an initial version of the complete survey instrument
composed of 79 items underwent face validation with acceptable
comprehensibility (2.96–4.00, out of 4.00) and relevance (0.77–1.00,
out of 1.00). Items were reduced to 77 after removal of two items
referring to hospital characteristics, not relevant to this study phase.
As for the focus group data, trustworthiness, credibility, and con-
firmability will be ensured by data triangulation from transcribed
audio recordings, field notes, and result of process evaluation, while
a reflective journal will serve as audit trail.
Changes in hospital factors will be accounted for by examining
the initial homogeneity between the two institutions on structural
and process variables. During the study implementation process,
field notes will be taken to document any significant changes in
operational issues such as human resource policies, unanticipated
change in hospital service use patterns due to natural disaster or
communicable disease outbreak etc. Setting the post-test evalua-
tion in multiple post-test time points also enhances the noise from
the contextual factors and increase the internal validity of this
study.
2.7.2 | Validation of the intervention protocol
The NET-CQI program is based on evidence-based strategies to im-
prove nurse autonomy and supporting nurses in the healthcare team
(TeamSTEPPS 2.0®, model for improvement) and informed by two
preliminary studies conducted by the investigators. Three experts re-
viewed the intervention protocol to validate its content and strategies,
relevance and appropriateness to the target setting, and adequacy of
information presented. All reviewers are experts in the field of nursing,
research, patient safety, and quality improvement and with experience
as Chief Nursing Officers of lead healthcare institutions.
2.8 | Data analysis
Data analysis will be done in the following sequence: (a) process
evaluation; (b) quantitative analysis; (c) qualitative analysis; and
(d) integration of the quantitative and qualitative results. Process
evaluation data will be first analysed to avoid biased interpretation
(Moore et al., 2015). This includes descriptive statistics reflecting
the intervention's fidelity, dose, and reach.
Generalized estimating equations (GEE) model will be used
to compare the change in primary outcomes of NWE, other out-
comes including job satisfaction, burnout risk, intention to leave
and perceived quality of care provided, and NWE-related vari-
ables of workload, time spent on nursing care and patient acuity
between the control and intervention groups. Comparison will be
made over three time points: baseline (T0), at 3  months of NET-
CQI implementation (T1), and post-intervention at 6 months (T2).
The time effect and group effect of these outcomes will also be re-
ported. All statistical tests involved will be two-sided with a level
of significance set at 0.05 and will be performed using IBM SPSS
22.0.
Audio recordings of the focus group interviews will be tran-
scribed in verbatim, while non-verbal expressions will be noted
based on the observer's field notes during the interviews. Content
analysis will be used. The two research team members and the
interviewer will be involved in data analysis. They will read the
transcriptions several times to generate initial codes for each cate-
gory of interviewees (intervention participants: staff and contrac-
tual nurses, nurse administrator; healthcare personnel: doctors
and non-intervention participant; patients) independently. More
discrete codes will be developed based on re-reading of tran-
scriptions and initial coding. A meeting will be held to discuss the
developed codes to ensure consensus. Categories will be gen-
erated from the codes, then, themes (Saldaña, 2016). Deductive
approach will be used to identify the patterns of meaning based
on the guide questions, while observing the rule of parsimony to
ensure the avoidance of overlapping categories (Braun & Clarke,
2006). Within-group analysis will be finalized to address the re-
search questions on the perception and experiences of nurses
who participated in the intervention and the impact of the pro-
gram on non-intervention participants (other healthcare workers,
PAGUIO et al.
administrators, and patients) respectively. Once within-group
themes have been determined, cross-group analysis will be done
to determine whether there are encompassing themes across par-
ticipant nurse groups, patients, and other healthcare professionals.
2.9 | Process evaluation
A process evaluation will be conducted to determine the extent of
implementation of NET-CQI, the compliance of participants in the
implementation process, and the contextual factors in the different
units on the outcomes measured in the study. The degree of inter-
vention implementation will be based on two sources: (a) the at-
tendance of participants in the activities scheduled in the two study
phases; and (b) field notes made by the TeamSTEPPS Master Trainers
and QI facilitator on the nurses’ engagement during trainings and
mentoring sessions. Compliance with the implementation process
will be evaluated based on the nurses’ adherence to the interven-
tion and achievement of QI milestones—defining the unit problem,
developing the QI plan, implementation of PDSA cycles and presen-
tation of their results. Contextual factors throughout the study im-
plementation period will be evaluated based on two data sources: (a)
unit characteristics taken throughout the data collection points; and
(b) field notes of co-investigators on hospital-level improvements or
changes affecting the nursing workforce.
3 |  D I S CU S S I O N
The study will use a quasi-experimental approach to test the effec-
tiveness of a program on improving NWE among nurses in matched
hospital units. This approach allows for testing interventions in real-
world settings, where random allocation is logistically impossible
and impractical because of the nature of healthcare environments
(Handley, Lyles, McCulloch, & Cattamanchi, 2018). In addition, a
qualitative approach will be embedded to complement the experi-
ment to provide a deeper understanding of the intervention effect
on the nurse and other healthcare providers and their care recipients
within the nurses’ sphere of influence.
A significant number of studies have attempted to improve NWE
and nurse outcomes through varied approaches (Schalk et al., 2010;
Wei et al., 2018). However, limitations in methodological presenta-
tion and presentation of contextual basis for its appropriateness in
resource-varied settings have prevented replication. This study pro-
tocol potentially addresses these research gaps by presenting the
contextual basis of the intervention and its potential application in
non-Western, high workload settings. In addition, the extensive pre-
sentation of the intervention and its procedures will facilitate repli-
cation to generate new evidence.
With the increasing importance of addressing NWE globally and
the urgency to address the unhealthy NWE in different settings,
this current study helps in generating new evidence that nurses can
apply in their own context. Results of this study may inform future
|
      673
strategies that address context-specific needs while generating
replicable approaches to improve nurse work environments, nurse
job satisfaction, staff retention, and quality care provision.
3.1 | Limitations
Considering the pre-experimental design and the implementation
in real-world settings, this study has several known and anticipated
limitations. These include potential confounding bias and ascertain-
ment bias and issues with compliance with intervention and subject
attrition.
3.1.1 | Biases due to confounders and outcome
measurement
Separating hospitals into intervention and control minimizes the con-
tamination across study units. Matching intervention units and par-
ticipants with control groups and having repeated measures pre- and
post-intervention addresses threats to the validity of lack of rand-
omization and address potential confounding bias (Handley, Beswick,
Goldmann, & Wallack, 2015). Blinding the outcome assessors at T2
and having a control group will minimize the ascertainment bias.
3.1.2 | Intervention compliance and subject attrition
Despite plans to ensure that participation in the study intervention
will not result in significant additional workload, the series of ac-
tivities requiring the nurses’ active involvement and institution-led
changes throughout the study implementation period may affect
the intervention's compliance (Van Bogaert et al., 2017). To over-
come this limitation, meticulous scheduling and close coordination
to accommodate nurse availability will be observed. Regular monthly
mentoring sessions will be done to motivate nurses to comply with
intervention protocol and remain in the study. Benefits of continu-
ous participation, such as continuing professional education units
that awarded at the end of the study and potential benefits to their
units will be emphasized.
3.2 | Ethical considerations
Ethical approval was obtained from a university ethics committee
(2018.590-T) and hospital research ethics review boards (RERC
2018-65; RERC 2018-21; RERC 2019-002) from Hong Kong and the
Philippines between December–March 2019. Written institutional
and individual consent will be secured. Anonymity and confiden-
tiality will be observed by ensuring that names of participants are
not reflected in data collection forms. Data collected will be kept
in secure location during the study period and accessible to only
the research team. Voluntary participation will be emphasized, and
 |
674       PAGUIO et al.
participants will be informed that they may withdraw from the study
at any time without any repercussions.
AC K N OW L E D G E M E N T S
The authors thank Prof. CHOI Kai Chow for statistical assistance.
C O N FL I C T O F I N T E R E S T
No conflict of interest was declared by the authors in relation to the
study itself. Note that Doris Yu is a JAN editor but, in line with usual
practice, this paper was subjected to double-blind peer review and
was edited by another editor.
ORCID
Jenniffer Torralba Paguio  https://orcid.org/0000-0001-5779-3580
Doris Sau Fung Yu  https://orcid.org/0000-0002-9359-1748
T WITTER
Jenniffer Torralba Paguio  @jtpaguioRN
Doris Sau Fung Yu  @DorisYu77098138
REFERENCES
Agency for Healthcare Research and Quality (AHRQ). (2019). Team strat-
egies and tools to enhance performance and patient safety 2.0 in-
structor manual. Retrieved from https​://www.ahrq.gov/teams​tepps/​
instr​uctor/​funda​menta​ls/index.html
Aiken, L., Buchan, J., Ball, J., & Rafferty, A. M. (2008). Transformative
impact of Magnet designation: England case study.
Journal of Clinical Nursing, 17(24), 3330–3337. https​://doi.
org/10.1111/j.1365-2702.2008.02640.x
Aiken, L. H., Sloane, D. M., Ball, J., Bruyneel, L., Rafferty, A. M., &
Griffiths, P. (2018). Patient satisfaction with hospital care and nurses
in England: An observational study. British Medical Journal Open, 8(1),
1–8. https​://doi.org/10.1136/bmjop​en-2017-019189
Aiken, L. H., Sloane, D., Griffiths, P., Rafferty, A. M., Bruyneel, L.,
McHugh, M., … Sermeus, W. (2016). Nursing skill mix in European
hospitals: Cross-sectional study of the association with mortality,
patient ratings and quality of care. BMJ Quality and Safety, 26(7),
559–568. https​://doi.org/10.1136/bmjqs-2016-005567
Alonso, A., Baker, D. P., Holtzman, A., Day, R., King, H., Toomey, L., &
Salas, E. (2006). Reducing medical error in the military health system:
How can team training help? Human Resource Management Review,
16(3), 396–415. https​://doi.org/10.1016/j.hrmr.2006.05.006
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychol-
ogy. Qualitative Research in Psychology, 3(2), 77–101. https​://doi.
org/10.1038/nrcar​dio.2016.5
Brewer, B. B., Wojner-Alexandrov, A. W., Triola, N., Pacini, C., Cline, M.,
Rust, J. E., & Kerfoot, K. (2007). AACN Synergy model's characteris-
tics of patients: Psychometric analyses in a tertiary care health sys-
tem. American Journal of Critical Care, 16(2), 158–167.
Ceravolo, D. J., Schwartz, D. G., Foltz-Ramos, K. M., & Castner, J.
(2012). Strengthening communication to overcome lateral vio-
lence. Journal of Nursing Management, 20(5), 599–606. https​://doi.
org/10.1111/j.1365-2834.2012.01402.x
Donabedian, A. (1988). The quality of care: How can it be assessed?
JAMA, 260(12), 1743–1748.
Donabedian, A. (2005). Evaluating the quality of medical care.
The Milbank Memoral Fund Quarterly, 44, 166–203. https​://doi.
org/10.1111/j.1468-0009.2005.00397.x
Gittell, J. H., Beswick, J., Goldmann, D., & Wallack, S. S. (2015). Teamwork
methods for accountable carecare: Relational coordination and
TeamSTEPPS®. Health Care Management Review, 40(2), 116–125.
https​://doi.org/10.1097/HMR.00000​0 0000​0 00021
Handley, M. A., Lyles, C., McCulloch, C., & Cattamanchi, A. (2018).
Selecting and improving quasi-experimental designs in effectiveness
and implementation research. Annual Review of Public Health, 39(1),
5–25. https​://doi.org/10.1146/annur​ev-publh​ealth-040617-014128
Institute for Healthcare Improvement (IHI) (2018). Quality improvement
essentials toolkit. Retrieved from http://www.ihi.org/resou​rces/
Pages/​Tools/​Q uali​t y-Impro​vement-Essen​tials-Toolk​it.aspx
International Council of Nurses. (2018). Evidence-based safe nurse staff-
ing. Geneva, Switzerland. Retrieved from https​://www.icn.ch/sites/​
defau​lt/files/​inline-files/​ICNPS​Evide​nceba​sedsa​fenur​sesta​f fing.pdf
Jeon, S. H., Park, M., Choi, K., & Kim, M. K. (2018). An ethical leader-
ship program for nursing unit managers. Nurse Education Today, 62,
30–35. https​://doi.org/10.1016/j.nedt.2017.12.017.
Kieft, R. A. M. M., De Brouwer, B. B. J. M., Francke, A. L., & Delnoij, D. M. J.
(2014). How nurses and their work environment affect patient expe-
riences of the quality of care: A qualitative study. BMC Health Services
Research, 14(1), 1–10. https​://doi.org/10.1186/1472-6963-14-249
Lake, E. T. (2002). Development of the practice environment scale of the
nursing work index. Research in Nursing and Health, 25(3), 176–188.
https​://doi.org/10.1002/nur.10032​
Lake, E. T. (2007). Measurement and evidence. Medical Care Research and
Review, 64(2), 104S–122S.
MacPhee, M., Dahinten, V. S., & Havaei, F. (2017). The impact of heavy per-
ceived nurse workloads on patient and nurse outcomes. Administration
Sciecne, 7(7), 1–17. https​://doi.org/10.3390/admsc​i7010007
McHugh, M., & Ma, C. (2014). Wage, work environment and staffing:
Effects on nurse outcomes. Policy, Politics and Nursing Practice, 15(3–
4), 72–80. https​://doi.org/10.2144/00011​4329.Funct​ional​
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman,
W., … Baird, J. (2015). Process evaluation of complex interventions:
Medical Research Council guidance. BMJ (Online), 350, 1–7. https​://
doi.org/10.1136/bmj.h1258​
Ogata, Y., Sasaki, M., Yumoto, Y., Yonekura, Y., Nagano, M., & Kanda, K.
(2018). Reliability and validity of the practice environment scale of
the nursing work index for Japanese hospital nurses. Nursing Open,
5(3), 362–369. https​://doi.org/10.1002/nop2.148
Paguio, J. T., & Yu, D. S. (2019). Diagnosing the nurse work environ-
ment of the Philippines: a countrywide study, International Council
of Nurses (ICN) 2019 International Conference. Marina Bay Sands,
Singapore.
Rao, A. D., Kumar, A., & McHugh, M. (2017). Better nurse autonomy de-
creases the odds of 30-day mortality and failure to rescue. Journal of
Nursing Scholarship, 49(1), 73–79. https​://doi.org/10.1111/jnu.12267​
Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.).
Los Angeles, CA: SAGE Publications.
Schalk, D. M., Bijl, M. L., Halfens, R. J., Hollands, L., & Cummings, G. G.
(2010). Interventions aimed at improving the nursing work environ-
ment: A systematic review. Implementation Science, 5(1), 1–11. https​://
doi.org/10.1186/1748-5908-5-34
Sexton, J. B., Helmreich, R. L., Neilands, T. B., Rowan, K., Vella, K.,
Boyden, J., … Thomas, E. J. (2006). The safety attitudes question-
naire: Psychometric properties, benchmarking data and emerg-
ing research. BMC Health Services Research, 6, 1–10. https​://doi.
org/10.1186/1472-6963-6-44
Stalpers, D., de Brouwer, B. J. M., Kaljouw, M. J., & Schuurmans, M. J.
(2015). Associations between characteristics of the nurse work en-
vironment and five nurse-sensitive patient outcomes in hospitals: A
systematic review of literature. International Journal of Nursing Studies,
52(4), 817–835. https​://doi.org/10.1016/j.ijnur​stu.2015.01.005
Turnipseed, D. L., & Turnipseed, P. H. (1997). A bi-cultural analysis of
the cost of caring: Nursing burnout in the United States and the
Philippines. Career Development International, 2(4), 180–188. https​://
doi.org/10.1108/13620​43971​0173670
PAGUIO et al. |
      675

Twigg, D., & McCullough, K. (2014). Nurse retention: A review of strate- Yoo, M. S., & Kim, K. J. (2017). Exploring the influence of nursing
gies to create and enhance positive practice environments in clinical work environment and patient safety culture on missed nursing
settings. International Journal of Nursing Studies, 51(1), 85–92. https​:// care in Korea. Asian Nursing Research, 47(9), 434–440. https​://doi.
doi.org/10.1016/j.ijnur​stu.2013.05.015 org/10.1097/NNA.00000​0 0000​0 00510
Van Bogaert, P., Van heusden, D., Verspuy, M., Wouters, K., Slootmans,
S., Van der Straeten, J., … White, M. (2017). The productive ward
program (TM): A two-year implementation impact review using a lon- S U P P O R T I N G I N FO R M AT I O N
gitudinal multilevel study. Canadian Journal of Nursing Research, 49(1),
Additional supporting information may be found online in the
28–38. https​://doi.org/10.1177/08445​62116​686491
Wei, H., Sewell, K. A., Woody, G., & Rose, M. A. (2018). The state of the Supporting Information section. 
science of nurse work environments in the United States: A system-
atic review. International Journal of Nursing Sciences, 5(3), 287–300.
https​://doi.org/10.1016/j.ijnss.2018.04.010 How to cite this article: Paguio JT, Yu DSF. A mixed methods
Weston, M. J. (2010). Strategies for enhancing autonomy and control
study to evaluate the effects of a teamwork enhancement and
over nursing practice. Olnine Journal of Issues in Nursing, 15(1), 1–31.
https​://doi.org/10.3912/OJIN.Vol15​No01M​an02 quality improvement initiative on nurses’ work environment. J
World Health Organization. (2016). Global strategy on human resources for Adv Nurs. 2020;76:664–675. https​://doi.org/10.1111/
health: Workforce 2030. Who, 64. Retrieved from http://apps.who.int/iris/ jan.14270​
bitst​ream/10665/​25036​8/1/97892​41511​131-eng.pdf?ua=1%5Cnht​
tp://www.who.int/hrh/resou​rces/pub_globs​trath​rh-2030/en/

The Journal of Advanced Nursing (JAN) is an international, peer-reviewed, scientific journal. JAN contributes to the advancement of evidence-based
nursing, midwifery and health care by disseminating high quality research and scholarship of contemporary relevance and with potential to ­advance
knowledge for practice, education, management or policy. JAN publishes research reviews, original research reports and methodological and
­theoretical papers.

For further information, please visit JAN on the Wiley Online Library website: www.wileyonlinelibrary.com/journal/jan

Reasons to publish your work in JAN:

• High-impact forum: the world’s most cited nursing journal, with an Impact Factor of 1.998 – ranked 12/114 in the 2016 ISI Journal Citation
Reports © (Nursing (Social Science)).
• Most read nursing journal in the world: over 3 million articles downloaded online per year and accessible in over 10,000 libraries worldwide
(including over 3,500 in developing countries with free or low cost access).
• Fast and easy online submission: online submission at http://mc.manuscriptcentral.com/jan.
• Positive publishing experience: rapid double-blind peer review with constructive feedback.
• Rapid online publication in five weeks: average time from final manuscript arriving in production to online publication.
• Online Open: the option to pay to make your article freely and openly accessible to non-subscribers upon publication on Wiley Online Library,
as well as the option to deposit the article in your own or your funding agency’s preferred archive (e.g. PubMed).