See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/260998833

Validation of a teamwork perceptions measure to increase patient safety

Article  in  BMJ quality & safety · March 2014

DOI: 10.1136/bmjqs-2013-001942 · Source: PubMed

CITATIONS READS

61 5,936

8 authors, including:

Joseph Roland Keebler Elizabeth Lazzara

Embry-Riddle Aeronautical University Embry-Riddle Aeronautical University
162 PUBLICATIONS   1,066 CITATIONS    94 PUBLICATIONS   3,550 CITATIONS   

SEE PROFILE SEE PROFILE

Lauren E. Benishek Heidi B King

Johns Hopkins Medicine United States Department of Defense
46 PUBLICATIONS   1,416 CITATIONS    39 PUBLICATIONS   3,466 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Combat Identification in Mixed Gender Teams View project

Using Escape Rooms for Teams Research View project

All content following this page was uploaded by Lauren E. Benishek on 15 March 2016.

The user has requested enhancement of the downloaded file.

 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
BMJ Quality & Safety Online First, published on 31 March 2014 as 10.1136/bmjqs-2013-001942
ORIGINAL RESEARCH

Validation of a teamwork
perceptions measure to increase
patient safety
Joseph R Keebler,1 Aaron S Dietz,2,3 Elizabeth H Lazzara,1,4
Lauren E Benishek,2,3 Sandra A Almeida,5 Phyllis A Toor,6 Heidi B King,6
Eduardo Salas2,3

1
Wichita State University,
Wichita, Kansas, USA
2
Institute for Simulation and
Training, University of Central
Florida, Orlando, Florida, USA
3
Department of Psychology,
University of Central Florida,
Orlando, Florida, USA
4
University of Kansas School of
Medicine Wichita, USA
5
Army Patient Safety Program,
US Army Medical Command,
Fort Sam Houston, Texas, USA
6
US Department of Defense
Patient Safety Program, Defense
Health Agency , Falls Church,
Virginia, USA
Correspondence to
Dr Eduardo Salas, Institute for
Simulation & Training, University
of Central Florida, 3100
Technology Parkway, Orlando,
FL 32826, USA;
esalas@ist.ucf.edu
Received 4 March 2013
Revised 17 January 2014
Accepted 28 February 2014
To cite: Keebler JR, Dietz AS,
Lazzara EH, et al. BMJ Qual
Saf Published Online First:
[ please include Day Month
Year] doi:10.1136/bmjqs-
2013-001942
ABSTRACT
Background TeamSTEPPS (Team Strategies and
Tools to Enhance Performance and Patient
Safety) is a team-training intervention which
shows promise in aiding the mitigation of
medical errors. This article examines the
construct validity of the TeamSTEPPS Teamwork
Perceptions Questionnaire (T-TPQ), a self-report
survey that examines multiple dimensions of
perceptions of teamwork within healthcare
settings.
Method Using survey-based methods, 1700
multidisciplinary healthcare professionals and
support staff were measured on their perceptions
of teamwork. Confirmatory factor analysis was
conducted to examine the relationship between
the five TeamSTEPPS dimensions: Leadership,
Mutual Support, Situation Monitoring,
Communication, and Team Structure.
Results The analysis indicated that the T-TPQ
measure is more reliable than previously thought
(Cronbach’s α=0.978). Further, our final tested
model showed a good fit with the data (x2 (df )
3601.27 (546), p<0.0001, Tucker–Lewis Index
(TLI)=0.942, Comparative fit index (CFI)=0.947,
root mean square error of approximation
(RMSEA)=0.057), indicating that the measure
appears to have construct validity. Further, all
dimensions correlated with one another, but
were shown to be independent constructs.
Conclusions The T-TPQ is a construct-valid
instrument for measuring perceptions of
teamwork. This has beneficial implications for
patient safety and future research that studies
medical teamwork.
BACKGROUND
Despite the growing national recognition
of the pivotal role of teamwork for safe,
quality healthcare, the educational curric-
ula and medical practices of few health
professions integrate team training. In
response to this critical gap, the
Department of Defense Patient Safety
Program (DoD PSP) in collaboration with
the Agency for Healthcare Research and
Quality (AHRQ) and a national team of
subject matter experts launched a multi-
year research and development effort to
create TeamSTEPPS (Team Strategies and
Tools to Enhance Performance and
Patient Safety), releasing the programme
in 2006 as a publicly available resource
and the AHRQ national standard for
medical team training.
Evidence of TeamSTEPPS’ effectiveness
across healthcare settings is beginning to
accumulate, with research studies showing
improvements in team skills such as lead-
ership, situation monitoring, mutual
support and communication,1 as well as
reductions in medical errors related to
communication, medication, needle-stick
incidents2 and endotracheal intubation.1
Given that every state has implemented
TeamSTEPPS and the Military Health
System is one of the largest in the world,
employing over 130 000 personnel and
serving 9.6 million beneficiaries,
TeamSTEPPS will probably sustain and
continue to broaden.
In parallel, the National Strategy for
Quality Improvement3 explicated measure-
ment as one of the primary objectives for
maximising results of patient safety
advancements. The health professions edu-
cation system has moved toward an out-
comes/competency-based system, requiring
that training demonstrates effectiveness by
measuring performance, including patient
safety competencies such as teamwork.
Without adequate measurement, there will
continue to be inaccurate estimates of
medical errors and a lack of significant cul-
tivations in patient safety initiatives.4
Despite the criticality of teamwork, there

Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942 1

Copyright Article author (or their employer) 2014. Produced by BMJ Publishing Group Ltd under licence.
 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
Original research
remains a paucity of validated metrics for team
performance.5
The TeamSTEPPS programme provides a survey-
based metric of perceptions of teamwork—the
TeamSTEPPS Teamwork Perceptions Questionnaire
(T-TPQ). Unquestionably, surveys have inherent draw-
backs, such as being susceptible to responder bias;
however, evaluating perceptions of patient safety
culture and teamwork does have merit. To illustrate,
Mardon et al6 found that hospitals with higher
patient safety culture survey scores had fewer compli-
cations and adverse events. In addition, Manser5 con-
ducted a systematic literature review and found that
perceptions of teamwork were related to the quality
of patient care. Further, one study suggested that posi-
tive teamwork perceptions were related to lower mor-
tality rates.7 When team members believe that the
climate exists for open communication, valued opi-
nions and trusted decisions, teams behave more effect-
ively and efficiently.8 9 Others suggest that sources of
errors include poor staff–staff and provider–patient
communication, multiple handoffs, rapid decision-
making, staff stress, fatigue and lack of appropriate
education and training.10
Unarguably, performance measurement is founda-
tional for learning and behavioural change, but is
often difficult to assess in the medical setting.11
Therefore, the T-TPQ measures an individual’s per-
ception of group-level teamwork skills within a
medical unit or department.12 It was developed to
align with the five core teamwork dimensions on
which TeamSTEPPS is based: Team Structure,
Leadership, Situation Monitoring, Mutual Support
and Communication. These dimensions were synthe-
sised from a comprehensive literature review of the
evidence based on teamwork, patient safety and team
training13 14 and represent the core teamwork compe-
tencies that most heavily affect team performance.15
These competencies are relevant to teams performing
in dynamic, complex environments where the conse-
quences of errors are high but the occurrence of
errors are low.16
The present paper contributes to the advancements
of patient safety, and helps to address a current need
by extending beyond previous work that described the
initial creation and psychometric testing of the
T-TPQ.12 In particular, it details the validation efforts
of the T-TPQ.
RATIONALE
While previous efforts have provided initial evidence
of the T-TPQ’s reliability,12 there are several reasons
underscoring the need for additional assessment of
both its reliability and construct validity. First, there is
insufficient evidence on its construct validity.
Therefore, this paper will focus on an empirical
approach to aid in establishing this. Throughout the
remainder of this article, the term ‘validity’ will
2 Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942
specifically refer to ‘construct validity’ unless other-
wise noted. Although previous efforts have demon-
strated that the T-TPQ is reliable (eg, Cronbach’s α for
each dimension ranged from 0.88 to 0.95), assessment
of the measure’s validity was limited in the initial val-
idation effort.12 Thus, it remains unclear how well the
T-TPQ measures its five teamwork dimensions.
Second, previous validation efforts relied on a rela-
tively small sample size of 169 participants.12 It has
been suggested that, at a minimum, validation studies
should include 200 participants.17 Third, the valid-
ation procedures adopted in previous investigations
were limited in their approach to assessing validity.
This study uses a more robust validation technique to
better establish the reliability and validity of the
T-TPQ through confirmatory factor analysis (CFA).18
CFA is a technique capable of describing exactly how
well each of the T-TPQ items measures the dimension
of teamwork that it is purported to measure. Lastly,
the sample used in the previous effort was relatively
homogeneous; 73.3% of participants were direct
patient care providers.12 The relative absence of diver-
sity from that sample limits the extent to which evi-
dence on the reliability and validity of the T-TPQ will
generalise to populations outside of direct patient care
providers.
METHOD
The T-TPQ includes 35 items that target the five core
teamwork dimensions discussed above (seven items
per dimension). Responses are scored by asking parti-
cipants to indicate their level of agreement with each
T-TPQ item (ie, strongly agree, agree, neutral, disagree
or strongly disagree). The US Army provided the data
for this study and granted permission for analysis and
publication of findings. The study sample excluded
any identifying information that could be linked to
participants. Staff members from the US Army
medical facilities across the USA (N=1700) completed
the T-TPQ (table 1). A CFA was conducted using
Analysis of Moment Structure (AMOS) V.19. CFA is
ideal for analyses involving large sample sizes and is
uniquely suited to providing a more accurate account
of the T-TPQ’s reliability and validity than what was
previously reported.12 For instance, each T-TPQ item
can be assessed on how well it actually measures the
dimension that it is purported to measure and
whether items explain variance for multiple dimen-
sions. This is deemed the model’s ‘fit’ and determines
how well the measure assesses what it is intended to
assess. Below we will discuss in more detail how the
goodness-of-fit for a model represents its validity.
CFA allows hypothesis testing of the validity with
which a test measures its underlying factors/con-
structs/dimensions. Specifically ‘in testing for the val-
idity of factorial structure for an assessment measure,
the researcher seeks to determine the extent to which
items designed to measure a particular factor (ie,
 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
Original research

Table 1 Staff positions for study sample dimensions are represented as ovals, and the indica-
Staff position Frequency Percentage
tors of each dimension (ie, each T-TPQ item) are
represented as rectangles. Each item is directly con-
Administration/management/executive staff/ 229 13.47 nected to the dimension it is supposed to measure,
commander
and the relationships among the TeamSTEPPS’ dimen-
Aerospace/biomedical engineering specialist 2 0.12
sions are depicted as double-headed arrows. As illu-
Assistant/clerk/secretary/administrative 188 11.06
technician strated in figure 1, for example, Communication is
Dental assistant 3 0.18
thought to be a unique and important TeamSTEPPS’
Diet technician 27 1.59
dimension, and indicators of Communication (Q29–
Q35 of the T-TPQ) should only measure and repre-
Dietician 18 1.06
sent Communication. Similarly, the correlations
Information technology (IT) services staff 34 2.00
between Communication and the other TeamSTEPPS’
Laboratory technician 67 3.94
dimensions are thought to relate to one another in
LVN/LPN 94 5.53
some way. In sum, the responses to the T-TPQ items
Medical technician 171 10.06
(ie, the rectangles) are shown to be the product of the
Nurse practitioner 36 2.12 underlying TeamSTEPPS’ dimension (ovals).
Other provider (clinical psychologist, 84 4.94 Sample data were entered to estimate model fit.20
optometrist, social worker, etc)
Essentially, a good fitting CFA model gives estimates
Other staff position 161 9.47
that are very close to what could be expected from
Patient care assistant/nursing aid 31 1.82
the actual study population (ie, extrapolating sample
Pharmacist 30 1.76
data to all individuals these data seek to account for).
Pharmacy technician 38 2.24 From this analysis, it can be concluded that: (1) the
Physical/occupational/speech therapist 17 1.00 TeamSTEPPS’ dimensions are, in fact, unique and
Physician assistant 26 1.53 meaningful; (2) the T-TPQ items consistently measure
Physician/attending physician 124 7.29 what they are supposed to measure (ie, does an item
Radiology technician 4 0.24 seeking to measure Communication measure that
Registered nurse 283 16.65 dimension and only that dimension?).
Resident/intern/medical student 30 1.76 Ultimately, five models were assessed. At each stage,
Respiratory therapist 3 0.18 post hoc modifications were made to improve the fit
Total 1700 100.00 of the model. Starting with the first theoretical model,
LPN, licensed practical nurse; LVN, licensed vocational nurse. parameter estimates and fit indices were evaluated to
determine how well the data fit. Specifically, the ana-
latent variable construct) actually do so’.19 The lyses evaluated several fit indices commonly used to
metric’s subscales represent the underlying latent interpret CFA findings.19 The rationale for this
factors (in this case—the five dimensions of team- approach was to demonstrate that our findings con-
work), and all items that are aimed at measuring that sistently demonstrate the same results across a variety
subscale should ‘load’ (ie, co-vary) with that factor of metrics. This process of model assessment answers
and only with that factor. Therefore, the question ‘is the fundamental question: is it a good model?19 21
this a good model?’, as tested through the methods of
CFA, is answered by how well the CFA model fits the RESULTS
data. The better the fit, the more we can say that the The validity of the T-TPQ was found through model-
model represents a metric that measures what it is ling procedures of CFA. That is, the set of seven ques-
intended to measure. A well-fitting model indicates tions that accompany each dimension clearly represent
that (a) the underlying factors account for the vari- that dimension. Also, the final analysed model demon-
ance in the items within their subscale (ie, questions strates that each of the dimensions is a justifiable com-
for a particular dimension only co-vary with that ponent of individual perceptions of teamwork. Many
dimension) and (b) the underlying factors do not of the dimensions are strongly related to one another
account for variance in other subscales. Further, it has (eg, Situation Monitoring and Mutual Support), which
been suggested that it’s only appropriate to use CFA is logical when one considers each dimension as an
when the metric in question has been (a) theoretically aspect of teamwork. As an example, it is impossible to
derived and (b) previously tested empirically.19 Both monitor a situation as a team without mutual support
of these requirements have been met for the T-TPQ, from one’s team members. Multiple individuals need
and, given that it has been implemented in situ in to cooperate for effective situation awareness and mon-
multiple operational environments (eg, hospitals), it itoring, and this is evident in the data. To assess the
appears to be a perfect candidate for validity testing strength of each model, three common fit indices were
using CFA. used: Tucker–Lewis Index (TLI), Comparative Fit
The analysis began by specifying a theoretical model Index (CFI) and root mean square error of approxima-
in AMOS. In this model, the TeamSTEPPS’ tion (RMSEA).19 Each of these fit indices has a set

Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942 3

 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
Original research

Figure 1 Final confirmatory factor analysis (CFA) model (model 5). Asterisks indicate significant relationships at p<0.001.

cut-off value, akin to the common p<0.05 of hypoth- ≥0.96 and RMSEA ≤0.06.22 Given this, the RMSEA is
esis testing. Specifically, when used in combinations (ie, commonly accepted as one of the best fit indices avail-
more than one fit index for a model), the acceptable able.23 Specifically, RMSEA provides a robust index of
values for these indices are as follows: TLI ≥0.95, CFI whether the tested model is of high quality, if the

4 Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942

 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
Figure 2 The five dimensions of teamwork measured by the TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ). The
construct of team structure is represented by the Patient Care Team encircling the other four dimensions.
model is mis-specified or not, as well as providing
a CI.23
Model 1: Our first model contained the five dimen-
sions and their representative questions and error
terms. Each dimension had seven associated questions.
This model showed a reasonable fit: x2 (df ) 4829.299
(550), p<0.0001, TLI=0.919, CFI=0.925, RMSEA=
0.068. Owing to high modification indices between
some of the error terms resulting from this procedure,
it was decided that a second model be analysed in
which the error terms for questions 29 and 31were
correlated, both of which were under the dimension
of Communication, and which contained the highest
modification index (410.369).
Model 2: After correlation of the error terms
between items 29 and 31, the resultant model had the
following fit indices: x2 (df ) 4375.639 (549),
p<0.0001, TLI=0.928, CFI=0.933, RMSEA=0.064.
Again, the modification indices were examined, and it
was observed that there was a high modification index
(305.513) between the error terms for questions 12
and 13, both under the dimension of Leadership.
Therefore, a third model was run which included a
correlated error term for these items.
Model 3: After correlation of the error term
between items 12 and 13, the resulting model had the
following fit indices: x2 (df ) 4043.456 (548),
p<0.0001, TLI=0.934, CFI=0.939, RMSEA=0.061.
Modification indices were again examined, and it was
found that the error terms between questions 26 and
27, which fall under the dimension of Mutual
Support, had a high modification index (297.472). A
fourth analysis was conducted with correlated error
terms for items 26 and 27.
Model 4: After correlation of the error term
between items 26 and 27, the resulting model had the
following fit indices: x2 (df ) 3724.847 (547),
p<0.0001, TLI=0.940, CFI=0.945, RMSEA=0.058.
Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942
Original research
This model shows a good fit, yet modification indices
demonstrated that the error terms for items 22 and
23, which were also both under Mutual Support, had
a high modification index (118.023). A fifth and final
analysis was conducted using a model including all
three previous error terms and the error term
between 22 and 23.
Model 5: This included four sets of correlated error
terms. All of these error terms were between items
within dimensions, and included: questions 12 and 13
under Leadership; questions 22 and 23 under Mutual
Support; questions 26 and 27 under Mutual Support;
and questions 29 and 31 under Communication.
Upon examination of the correlated items, it was
apparent that these questions contained highly similar
content (eg, staff ‘speak up’ when they have a concern
about the patient), which would therefore lead to
their correlated errors. The fit indices for this final
model were good (x2 (df ) 3601.27 (546), p<0.0001,
TLI=0.942, CFI=0.947, RMSEA=0.057). The final
model is shown in figure 2, and the correlations
between latent variables are shown in table 2.
The hypothesised model containing five dimensions
(Team Structure, Leadership, Situation Monitoring,
Mutual Support and Communication) fits the data
very well. In other words, the theoretical model we
set out to validate was predicted by the 1700 data
Table 2 Estimated correlations between dimensions (N=1700)
Variable LD SM MS COMM
TS 0.78* 0.87* 0.85* 0.77*
LD 0.71* 0.72* 0.67*
SM 0.91* 0.84*
MS 0.87*
*p<0.001.
COMM, Communication; LD, Leadership; MS, Mutual Support; SM,
Situation Monitoring.
5
 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
Original research
Table 3 CFA fit indices for each model.
CFA index
standards19 Model 1
2
x (df) – 4829.299 (550),
p<0.0001
TLI x>0.9 0.919
CFI x>0.9 0.925
RMSEA x<0.8 0.068
Modification – Items 29 and 31:
Index (410.369)
CFA, confirmatory factor analysis; CFI, Comparative Fit Index; RMSEA, root mean square error of approximation; TLI, Tucker–Lewis Index.
points we collected from healthcare workers. The
final model clearly demonstrated that all five dimen-
sions of the T-TPQ are important components for
measuring individual perceptions of teamwork.
Specifically, all fit indices within our final model (ie,
Model 5) demonstrated borderline to good fit given
index standards in CFA: TLI=0.942 (≥0.95),
CFI=0.947 (≥0.96) and RMSEA=0.057 (≤0.06; CI
90=0.056–0.059)22 (see table 3 for a summary of all
fit indices and acceptable value ranges). Also, and just
as important, measures of one dimension did not cor-
relate with or predict other dimensions. Therefore,
each of the dimensions is an independent, yet integral,
component of the T-TPQ. Given this outcome, we can
posit with certainty that the T-TPQ can be used as a
valid measure in healthcare settings. Below, the reli-
ability (the consistency with which the measured vari-
ables are assessed by the T-TPQ) and validity (whether
the measure is representing the dimensions or not) of
the T-TPQ will be reviewed.
The overall reliability of the T-TPQ was excellent
(Cronbach’s α=0.978). Compared with previous reli-
ability analyses, this large sample analysis demon-
strates a higher reliability than previously concluded
(see figure 2 and table 4). Also, the reliability of the
individual dimensions was exceptional, and each
exceeded an acceptable level of 0.9 or more. The reli-
abilities for each dimension are listed in table 4. Given
these reliability estimates, it seems that the T-TPQ has
a very high internal consistency (ie, the items are
related in what they measure) and is therefore a reli-
able measure of individual perceptions of teamwork.
When the outcome of this CFA is examined in rela-
tion to the previous T-TPQ analysis, it is clear that the
reliability of the measure is better than previously sus-
pected (see figure 2). Specifically, the seven items used
to measure each of the five dimensions are, in fact,
measuring their targeted teamwork dimension. Also,
none of the items cross-correlate, which indicates that
none of the T-TPQ items measure more than one
dimension. This point is important and noteworthy
because, if items measure multiple dimensions, it can
be argued that the number of dimensions is incorrect
or needs to be reduced. There is no basis for this phe-
nomenon concerning the metric being discussed here.
6 Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942
Model 2 Model 3 Model 4 Model 5
4375.639 (549), 4043.456 (548), 3724.847 (547), 3601.270 (546),
p<0.0001 p<0.0001 p<0.0001 p<0.0001
0.928 0.934 0.940 0.942
0.933 0.939 0.945 0.947
0.064 0.061 0.058 0.057
Items 12 and 13: Items 26 and 27: Items 22 and 23: N/A
(305.513) (297.472) (118.023)
Therefore, according to our analysis, it can be con-
cluded from these data that all five dimensions are
important facets of teamwork, and that, across 1700
healthcare professionals and support staff, all five
dimensions are consistent components of individuals’
perceptions of teamwork.
DISCUSSION
This effort was a large sample follow-up to previous
reliability analyses that used a much smaller sample
size12 and adds to the previous work by showing that
the T-TPQ is more reliable and that it demonstrates
validity, compared with previous analyses. Our CFA
model demonstrates a strong case that the T-TPQ is
actually measuring its intended dimensions. Further,
the CFA demonstrated remarkable fit across multiple
standardised indices. These fit indices provide clear
support that the T-TPQ can be used as a valid tool to
measure individual staff members’ perceptions of
group-level teamwork within their unit or department
across multiple healthcare settings, professions and
work positions. Also, reliability analyses demonstrated
high reliability for both the entire measure and each
of its sub-dimensions.
Although the analyses presented here tested the
structure of the T-TPQ (ie, construct validity) as a
measurement tool of perceptions of teamwork, they in
fact did not test ‘criterion validity’ or ‘predictive valid-
ity’. In other words, this analysis only shows to what
extent the survey measured perceptions of teamwork
and does not directly link the perceptions of teamwork
to external performance criteria (eg, patient safety out-
comes). Other research has shown that good teamwork
is associated with increased patient safety, specifically
lower mortality rates,24 but this paper does not
provide empirical data demonstrating that relationship.
Given these results, this CFA demonstrated that the
T-TPQ is: (a) a valid measure, at least construct-wise,
of its individual dimensions; (b) reliable, both as an
overall survey and at the level of each dimension.
Assessing teamwork is an integral component of
patient safety. Without proper teamwork, it is diffi-
cult, if not impossible, to increase patient safety out-
comes.25 The T-TPQ is a valid and reliable measure of
individual staff members’ perceptions of the quality of
 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
Original research

Table 4 Summary of reliability for T-TPQ items and dimensions

Previous
Corrected item Cronbach’s α if reliability
No Item total correlation item deleted Dimension α from AHRQ
1 Skill of staff overlap sufficiently for work to be shared 0.624 0.916 Team Structure 0.917
when necessary
2 Staff are held accountable for their actions 0.704 0.909
3 Staff within my unit share information that enables 0.785 0.900 0.89
timely decision making by the direct patient care team
4 My unit makes efficient use of resources 0.768 0.902
5 Staff understand their roles and responsibilities 0.792 0.900
6 My unit has clearly articulated goals 0.769 0.902
7 My unit operates at a high level of efficiency 0.778 0.900
8 My supervisor/manager considers staff input when 0.869 0.949 Leadership 0.957
making decisions about patient care
9 My supervisor/manager provides opportunities to 0.860 0.950 0.95
discuss the unit’s performance after an event
10 My supervisor/manager takes time to meet with staff to 0.862 0.950
develop a plan for patient care
11 My supervisor/manager ensures that adequate resources 0.803 0.954
are available
12 My supervisor/manager resolves conflicts successfully 0.847 0.951
13 My supervisor/manager models appropriate team 0.865 0.949
behaviour
14 My supervisor/manager ensures that staff are aware of 0.854 0.950
any situations or changes that may affect patient care
15 Staff effectively anticipate each other’s needs 0.798 0.935 Situation 0.943
16 Staff monitor each other’s performance 0.754 0.939 Monitoring
17 Staff exchange relevant information as it becomes 0.828 0.932 0.91
available
18 Staff continuously scan the environment for important 0.852 0.930
information
19 Staff share information regarding potential 0.830 0.932
complications
20 Staff meet to re-evaluate patient care goals when 0.816 0.933
aspects of the situation have changed
21 Staff correct each other’s mistakes to ensure that 0.793 0.935
procedures are followed properly
22 Staff assist fellow staff during high workload 0.737 0.910 Mutual Support 0.920
23 Staff request assistance from fellow staff when they feel 0.727 0.910
overwhelmed
24 Staff caution each other about potentially dangerous 0.778 0.907 0.90
situations
25 Feedback between staff is delivered in a way that 0.803 0.902
promotes positive interactions and future change
26 Staff advocate for patients even when their opinion 0.729 0.910
conflicts with that of a senior member of the unit
27 When staff have a concern about patient safety, they 0.756 0.908
challenge others until they are sure the concern has
been heard
28 Staff resolve their conflicts, even when the conflicts 0.754 0.908
have become personal
29 Information on patient care is explained to patients and 0.776 0.932 Communication 0.939
their families in lay terms
30 Staff relay relevant information in a timely manner 0.813 0.929
31 When communicating with patients, staff allow enough 0.829 0.927 0.88
time for questions
Continued

Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942 7

 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
Original research
Table 4 Continued
No Item
32 Staff use common terminology when communicating
with each other
33 Staff verbally verify information that they receive from 0.828
one another
34 Staff follow a standardised method of sharing
information when handing off patients
35 Staff seek information from all available sources
AHRQ, Agency for Healthcare Research and Quality; T-TPQ, TeamSTEPPS Teamwork Perceptions Questionnaire.
teamwork within their work units. Although it does
not measure actual teamwork behaviours, it is a rela-
tively time- and cost-effective survey that can evaluate
medical team members’ perceptions: Team Structure,
Leadership, Situation Monitoring, Mutual Support
and Communication. In addition, the T-TPQ is a
viable alternative for gaining insights into teamwork
when assessment of teamwork behaviours (eg, atti-
tudes, behaviours and cognitions) is impractical or
impossible. When used in addition to objective mea-
sures of teamwork behaviours, such as direct observa-
tions, the T-TPQ can be a valuable component of a
comprehensive teamwork assessment and
performance-improvement strategy. For those health-
care organisations that have implemented
TeamSTEPPS, the T-TPQ is a construct-valid tool for
assessing and continually improving TeamSTEPPS’
training, implementation and sustainment. Further,
the T-TPQ would probably also be a useful tool for
evaluating other performance-improvement interven-
tions that focus on enhancing these specific teamwork
dimensions. Conversely, because it is designed to
assess just this set of five teamwork dimensions, the
T-TPQ may not be appropriate for assessment of inter-
ventions that are aimed at improving other facets of
teamwork (eg, collective efficacy and team psycho-
logical safety). In these instances, the T-TPQ may
underestimate the intervention impact because of
incongruences between the teamwork dimensions tar-
geted by the intervention and those measured by the
T-TPQ. In short, users should consider these to be
potential limitations for assessing whether the T-TPQ
is an appropriate measure for evaluating their inter-
ventions; otherwise they may make erroneous judge-
ments about the interventions’ effectiveness.
Practically speaking, the T-TPQ is a short, construct-
valid survey that can aid frontline clinicians, patient
safety professionals, researchers, medical educators
and healthcare administrators in gathering insightful
information to evaluate teamwork and the associated
safety climate within their medical units and depart-
ments. This survey provides a reliable and (construct)
valid instrument to assess teamwork perceptions based
on a prevalent training, which is a worthwhile step in
improving team-training initiatives, teamwork and
8 Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942
Previous
Corrected item Cronbach’s α if reliability
total correlation item deleted Dimension α from AHRQ
0.807 0.930
0.927
0.764 0.933
0.793 0.930
ultimately patient safety. Owing to the difficulty of
collecting medical professional behaviours in situ, a
valid survey that can assess perceptions in proxy of
actual behaviours allows researchers to understand
major dimensions of teamwork in an easily accessible,
non-intrusive way. Future research is needed to
further understand the T-TPQ in relation to relevant
patient safety outcomes such as safety culture, patient
harm and medical errors.
Correction notice This article has been corrected since it was
published Online First. Figure legends 1 and 2 have been
transposed.
Contributors Each author provided a unique and needed insight
and effort into the completion of this manuscript, from original
conception through iterative editing and final proofing. JRK
provided a substantial contribution through analysis and
interpretation of the data, outlining and drafting the article, and
providing tasking and substantial writing of portions of the
manuscript. ASD contributed through aiding in all aspects of
the analysis, as well as writing portions of the method and
conducting portions of the literature review. EHL contributed
through writing major portions of the introduction, conducting
portions of the literature review, and providing expertise in
AMA formatting, and creation of many of the tables and
figures. LEB contributed through providing major portions of
writing of the rationale section, and through writing portions of
the method. PAT, SAA and HBK were responsible for
conception, design and acquisition of the datasets. They all
contributed substantially to both initial writing efforts, many of
the following edits, and handling the manuscript as it passed
through the DoD approval process, which was a necessary and
vital step to be able to publish these data. They were also
responsible for giving access to the data and ensuring that the
manuscript was a significant contribution to the TeamSTEPPS
programme and patient safety in general. ES provided expert
management and guidance of the writing team through his
expertise in teamwork. ES reviewed, contributed to and edited
many iterations of the manuscript. Further, he provided
invaluable insights on measurements of teams, and aided as a
liaison between the Army Patient Safety programme and the
University of Central Florida throughout the entire process of
writing the manuscript.
Funding This publication was prepared by Booz Allen Hamilton
under contract to TRICARE Management Activity, Department
of Defense (DoD) Contract No W81XWH-08-D-0025, Task
Order No 0015. The views herein are those of the authors and
are not to be construed as official or as reflecting the views of
TRICARE Management Activity or the Department of Defense.
Competing interests None.
Ethics approval University of Central Florida, US Army, US
Department of Defense.
Provenance and peer review Not commissioned; externally
peer reviewed.
 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com
REFERENCES
1 Capella J, Smith S, Philip A, et al. Teamwork training improves
the clinical care of trauma patients. J Surg Educ
2010;67:439–43.
2 Deering S, Rosen M, Ludi V, et al. On the front lines of patient
safety: implementation and evaluation of team training in Iraq.
J Qual Patient Saf 2011;37:250–7.
3 U.S. Department of Health and Human Services. National
Strategy Quality Improvement in Health Care 2012 Annual
Progress Report to Congress. http://www.ahrq.gov/
workingforquality/nqs/nqs2012annlrpt.pdf Published 2012.
(accessed 10 Jun 2012).
4 Rosen AK. Are we getting better at measuring patient safety?
Agency for Healthcare Research and Quality website. http://
webmm.ahrq.gov/perspective.aspx?perspectiveID=94
Published November 2010. (accessed 9 May 2012).
5 Manser T. Teamwork and patient safety in dynamic domains of
healthcare: a review of the literature. Acta Anaesthesiol Scand
2009;53:143–51.
6 Mardon R, Khanna K, Sorra J, et al. Exploring relationships
between hospital patient safety culture and adverse events. J
Patient Saf 2010;6:226–32.
7 Wheelan SA, Burchill CN, Tilin F. The link between teamwork
and patients’ outcomes in intensive care units. Am J Crit Care
2003;12:527–34.
8 Edmondson AC. Psychological safety and learning behavior in
work teams. Adm Sci Q 1999;44:350–83.
9 Hackman JR, Oldham GR. Motivation through the design of
work: test of a theory. Organ Behav Hum Perform
1976;16:250–79.
10 Leape LL. Error in medicine. JAMA 1994;272:1851–7.
11 Galvin RS, McGlynn E. Using performance measurement to
drive improvement: a road map for change. Med Care
2003;41:48–60.
12 American Institutes for Research. TeamSTEPPS teamwork
perceptions Questionnaire (T-TPQ) manual. Washington, DC:
American Institutes for Research, 2010.
Keebler JR, et al. BMJ Qual Saf 2014;0:1–9. doi:10.1136/bmjqs-2013-001942
Original research
13 Baker DP, Gustafson S, Beaubien J, et al. Medical teamwork
and patient safety: the evidence-based relation. Washington,
DC: American Institutes for Research, 2003.
14 Baker SP, Salas E, Battles JB, et al. The relation between
teamwork and patient safety. In: Carayon P. ed. Handbook of
human factors and ergonomics in health care and patient safety.
2nd edn. Boca Raton, FL: CRC Press, 2012:185–98.
15 Salas E, Sims DE, Burke CS. Is there a “Big Five” in
teamwork? Small Gr Res 2005;36:555–99.
16 Baker DP, Day R, Salas E. Teamwork as an essential component
of high-reliability organizations. Health Serv Res
2006;41:1576–98.
17 Gorsuch RL. Factor analysis. 2nd edn. Hillsdale, NJ: Erlbaum,
2003.
18 Blunch NJ. Introduction to structural equation modeling using
SPSS and AMOS. Thousand Oaks, CA: Sage Publications LTD,
2008.
19 Byrne BM. Structural equation modeling with AMOS. 2nd edn.
New York, NY: Routledge, 2010.
20 Kohn LT, Corrigan JM, Donaldson MS. eds. To err is human:
building a safer health system. Committee on Quality of Health
Care in America, Institute of Medicine. Washington, DC:
National Academy Press, 2000.
21 Tabachnick BG, Fidell LS. Using multivariate statistics. Boston,
MA: Allyn & Bacon, 2001.
22 Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance
structure analysis: conventional criteria versus new alternatives.
Struct Equ Modeling 1999;6:1–55.
23 MacCallum RC, Austin JT. Applications of structural equation
modeling in psychological research. Annu Rev of Psychol
2000;51:201–26.
24 Neily J, Mills PD, Young-Xu Y, et al. Association between
implementation of a medical team training program and
surgical mortality. JAMA 2010;304:1693–700.
25 Firth-Cozens J. Cultures for improving patient safety through
learning: the role of teamwork. Qual Health Care 2001;10:
26–31.
9
 Downloaded from qualitysafety.bmj.com on June 5, 2014 - Published by group.bmj.com

Validation of a teamwork perceptions

measure to increase patient safety
Joseph R Keebler, Aaron S Dietz, Elizabeth H Lazzara, et al.

BMJ Qual Saf published online March 20, 2014

doi: 10.1136/bmjqs-2013-001942

Updated information and services can be found at:

http://qualitysafety.bmj.com/content/early/2014/03/31/bmjqs-2013-001942.full.html

These include:
References This article cites 15 articles, 1 of which can be accessed free at:
http://qualitysafety.bmj.com/content/early/2014/03/31/bmjqs-2013-001942.full.html#ref-list-1

P<P Published online March 20, 2014 in advance of the print journal.

Email alerting Receive free email alerts when new articles cite this article. Sign up in
service the box at the top right corner of the online article.

Notes

Advance online articles have been peer reviewed, accepted for publication, edited and
typeset, but have not not yet appeared in the paper journal. Advance online articles are
citable and establish publication priority; they are indexed by PubMed from initial
publication. Citations to Advance online articles must include the digital object identifier
(DOIs) and date of initial publication.

To request permissions go to:

http://group.bmj.com/group/rights-licensing/permissions

To order reprints go to:

http://journals.bmj.com/cgi/reprintform

To subscribe to BMJ go to:

http://group.bmj.com/subscribe/

View publication stats