ORIGINAL ARTICLE

Development of an instrument for the evaluation of advanced

life support performance
L.-M. Peltonen1,2 €1,5 and M. Tommila6,7
, V. Peltonen3,4, S. Salantera
1
Department of Nursing Science, University of Turku, Turku, Finland
2
Intensive Care Unit, Turku University Hospital, Turku, Finland
3
Department of Clinical Medicine, University of Turku, Turku, Finland
4
Department of Anesthesia and Intensive Care, Satakunta Central Hospital, Pori, Finland
5
Developmental Services, Turku University Hospital, Turku, Finland
6
Division of Perioperative Services, Intensive Care Medicine and Pain Management, Turku University Hospital, Turku, Finland
7
Department of Anesthesiology and Intensive Care, University of Turku, Turku, Finland

Correspondence Background: Assessing advanced life support (ALS) competence

L.-M. Peltonen, Department of Nursing
Science, 20014 University of Turku, Finland
E-mail: laura-maria.peltonen@utu.fi
Conflicts of interest
None declared.
Funding
This study project received support from the
Finnish Medical Society Duodecim, Turunmaa
Chapter.
Submitted 8 July 2017; accepted 21 July 2017;
submission 28 February 2017.
Citation
Peltonen L-M, Peltonen V, Salanter€a S,
Tommila M. Development of an instrument for
the evaluation of advanced life support
performance. Acta Anaesthesiologica
Scandinavica 2017
doi: 10.1111/aas.12960
requires validated instruments. Existing instruments include aspects
of technical skills (TS), non-technical skills (NTS) or both, but one
instrument for detailed assessment that suits all resuscitation situa-
tions is lacking. This study aimed to develop an instrument for the
evaluation of the overall ALS performance of the whole team.
Methods: This instrument development study had four phases.
First, we reviewed literature and resuscitation guidelines to explore
items to include in the instrument. Thereafter, we interviewed
resuscitation team professionals (n = 66), using the critical incident
technique, to determine possible additional aspects associated with
the performance of ALS. Second, we developed an instrument
based on the findings. Third, we used an expert panel (n = 20) to
assess the validity of the developed instrument. Finally, we revised
the instrument based on the experts’ comments and tested it with
six experts who evaluated 22 video recorded resuscitations.
Results: The final version of the developed instrument had 69
items divided into adherence to guidelines (28 items), clinical deci-
sion-making (5 items), workload management (12 items), team
behaviour (8 items), information management (6 items), patient
integrity and consideration of laymen (4 items) and work routines (6
items). The Cronbach’s a values were good, and strong correlations
between the overall performance and the instrument were observed.
Conclusion: The instrument may be useful for detailed assessment
of the team’s overall performance, but the numerous items make
the use demanding. The instrument is still under development, and
more research is needed to determine its psychometric properties.

Editorial Comment
Resuscitation contains technical, non-technical, individual and team components. This study
developed a proto-type of an inclusive instrument to assess the overall performance of a team. Val-
idation will be needed before clinical use.

Acta Anaesthesiologica Scandinavica (2017)

ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd 1
 L.-M. PELTONEN ET AL.
High-quality advanced life support (ALS) is
fundamental to improving resuscitation out-
comes. ALS performance is often divided into
two aspects, technical skills (TS) and non-
technical skills (NTS) in the literature. TS refer
to actions such as chest compression, securing
the airway and delivering shock, while NTS are
often associated with teamwork, leadership and
communication aspects.
Excellent resuscitation skills are needed for
professionals providing ALS. CPR competency
may be defined as the cognitive knowledge and
psychomotor skills necessary for effective perfor-
mance in cardiac arrest situations.1 Theoretical
knowledge and an understanding of resuscita-
tion guidelines are crucial, but implementing
this knowledge through TS is as important.
However, the NTS also become essential as the
overall ALS performance is based on the result
of the activities and cooperation between several
actors. Still, professionals have been shown to
lack important ALS knowledge.2,3 Research fur-
ther shows that ALS competency rapidly decays
after courses,4,5 and that current assessment
methods of competency are problematic.4 Valid
assessment methods are essential to support the
development and maintenance of professionals’
ALS competence.
Resuscitation competency is usually assessed
in several ways, for example, with simulated
scenarios to assess practical skills and with
written knowledge tests to assess theoretical
knowledge.5,6 Integration of theoretical knowl-
edge into practice is often evaluated using an
objective structured clinical examination
(OSCE).7 OSCEs are developed based on speci-
fic scenarios. One problem with scenario-based
checklists is that they are not directly trans-
ferrable to other settings. To date, a vast variety
of checklists, tools and instruments for the eval-
uation of resuscitation performance exist,7,8
although significant levels of variability be-
tween evaluators still occur.9 Assessing the
development and maintenance of resuscitation
competence requires objective and validated
instruments that measure the important aspects
of resuscitation performance. To date, an overall
assessment of ALS performance is difficult as
validated instruments only cover some aspects
of ALS performance while other important
aspects remain disregarded. Exploring all
2 ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
important aspects associated with ALS is
important to improve performance. Therefore, the
aim of this study was to develop an instrument
for the evaluation of the overall ALS performance
of the whole team. Such an instrument could be
used in research and education.
Methods
This instrument development study was con-
ducted in four phases. First, we explored exist-
ing validated instruments developed for the
assessment of resuscitation performance through
a literature review and read resuscitation guide-
lines to explore items to include in the instru-
ment under development. Then, we interviewed
resuscitation team professionals to explore pos-
sible further aspects that should be included in
an overall assessment of the ALS performance.
Second, we developed an instrument based on
findings in phase one. Third, we used an expert
panel to assess the content validity of the devel-
oped instrument. Fourth, we modified the
instrument based on the experts’ feedback, and
finally, the instrument was tested on video
recorded cardiopulmonary resuscitations. The
study design is illustrated in Fig. 1.
Literature review
We systematically conducted a literature review
to explore validated instruments developed for
the evaluation of ALS performance. We searched
four databases (PubMed, Cinahl, Scopus and
the Web of Science) with combinations of the
terms: cardiopulmonary resuscitation, resuscita-
tion, CPR, cardiac arrest, heart arrest, advanced
cardiac life support, ACLS, advanced life sup-
port, ALS, team, emergency team, resuscitation
team, intensive care team, rapid response team,
cardiac team, competence, skill, non-technical,
technical, communication, cooperation, team-
work, team, leader, leadership, knowledge,
assessment, evaluation, performance, scale,
instrument, measurement, tool, score, test and
validity.
We excluded scenario-based checklists,
trauma resuscitation instruments and instru-
ments developed for neonatal resuscitation. We
further searched reference lists to find additional
instruments. The search resulted in 1406
Acta Anaesthesiologica Scandinavica (2017)

Fig. 1. Research design. This figure illustrates the design of the study including details for each of the five phases.
articles. Finally, 15 instruments10–25 were
included in the review. A flow chart of the arti-
cle selection process is integrated into Fig. 1.
Individual items were extracted from the
instruments, which were identified through the
literature review. These items were gathered
into a list of suggested items for the new instru-
ment for the evaluation of the overall ALS per-
formance of the whole team. Thereafter,
resuscitation guidelines (the European Resusci-
tation Council guidelines and the American
Heart Association guidelines) were assessed for
additional items to add to this list.
Interviews
We conducted interviews to explore what other
factors could be associated with the success of
ALS performance, according to resuscitation
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
ALS INSTRUMENT DEVELOPMENT
team professionals. Informants included nurses,
physicians and orderlies who worked in one
university hospital’s resuscitation team. This
team was assembled from experienced person-
nel of the hospital’s intensive care unit. They
had gone through in-service training to be a part
of the hospital resuscitation team. Resuscitation
simulation training was offered to them every
week, but only a few could participate at a time.
Hence, each resuscitation team member attended
the training once a month at the best.
Data were collected by means of the critical
incident technique,26 which can be used to
gather important information in well-defined
situations, such as resuscitations. The infor-
mants were interviewed as soon as possible
after a resuscitation by phone. All members par-
ticipating in the resuscitation team’s outreach
activity during the data collection period, who
3
 4

L.-M. PELTONEN ET AL.

Table 1 Instruments developed for the assessment of resuscitation performance.

Non-technical Technical Overall Inter-rater Internal Construct Content Criterion

Authors (year) Instrument skills skills score reliability consistency Stability validity validity validity

Andersen Tool for measurement of performance in + + + +

et al. (2010)10 multi-professional resuscitation teams
Brett-Fleegler Tool for resuscitation assessment using + + + +
et al. (2008)11 computerised simulation (TRACS)
Carmona Questionnaire on nurses’ knowledge + + +
Torres (2014)12 regarding the current CPR
recommendations 2010
Cooper & Adapted leadership behaviour description + + +
Wakelam (1999)13 questionnaire (LBDQ) for leadership skills
in a resuscitation team
Cooper & Cant (2014),14 Team emergency assessment measure + + + + + + + +
Cooper et al. (2014, (TEAM)
2016)15
Duff et al. (2013)16 Multiple-choice examination assessing + + + +
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd

cognitive and behavioural knowledge of

paediatric resuscitation
Grant et al. (2012)17 Evaluation tool for paediatric resident + + + + + +
competence in leading simulated
paediatric resuscitations
Kim et al. (2006) 18
The Ottawa Crisis Resource Management + + + + + + +
Global Rating Scale (Ottawa GRS)
Mancini & The objective criterion-referenced tool for + + +
Kaye (1985)19 evaluation of ACLS performance
Reid et al. (2012)20 Simulation team assessment tool (STAT) for + + + + +
paediatric resuscitations
Roh et al. (2012)21 Resuscitation self-efficacy scale for nurses + + + + +
Seaman Performance and Outcome Performance + +
et al. (1986) 22 measure (STARS)
Turner Visual analogue scale as an instrument to + +
Acta Anaesthesiologica Scandinavica (2017)

et al. (2008)23 measure self-efficacy in resuscitation skills

Walker Observational skill based clinical assessment + + + +
et al. (2011)24 tool for resuscitation (OSCAR)
Zajano Self-evaluation survey of teamwork and task + + + +
et al. (2014)25 load among medical providers (STTaMP)

Aspect of advanced life support skills, validity and reliability of each instrument and the number of items included in each instrument are displayed.

were willing to participate in the study, were
interviewed individually. The interviews were
structured and all interviews were conducted by
the same researcher. This was a physician with
expertise in the area. All informants were asked
two open-ended questions. First, to describe
anything that they could think of that was asso-
ciated with the success of ALS, and second, to
describe anything that could have been
improved. The interviewer wrote down the
responses during the interviews to support the
analysis. Data were collected between Septem-
ber 2013 and January 2014.
In keeping with Flanagans26 definition, we
defined critical incidents as events, where activi-
ties that are effective in supporting or impeding
successful resuscitation performance may be
observed. The unit of analysis was the smallest
possible phrase that could be extracted without
losing the context of the content. Data were
analysed with thematic content analysis.27 First,
the data were read through; interconnected
expressions were then assembled into meaning
units and interpreted based on their underlying
meaning. The meaning units were then screened
through for possible items to add to the list of
suggested items for the instrument under devel-
opment. Thereafter, subthemes were formed of
similar interpreted meaning units. Finally, the
subthemes were organised into main themes.
These main themes were then used to sort the
items in the list of suggested items for the
instrument under development. All researchers
participated in analysing the data and selecting
items for the instrument.
Expert panel
A preliminary version of the instrument for the
evaluation of overall ALS performance was
developed based on findings from the literature
review, the resuscitation guidelines and the
interviews. An expert panel was used to assess
the content validity of the developed instru-
ment. We used snowball sampling to reach
experts for our panel. They were physicians and
nurses, who were recognised by peers and had
a substantial experience of resuscitations. We
reached for experts with different specialisa-
tions. We further targeted experts with work
experience from different clinical settings and
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
ALS INSTRUMENT DEVELOPMENT
from educational institutions. The panel
consisted of 20 experts. Individual items were
assessed using the content validity index
(CVI).28,29 Each item was rated by an expert on
a scale from 1 to 4 (1 = not relevant, 2 = some-
what relevant, 3 = quite relevant and 4 = highly
relevant). Ratings of 1 and 2 were considered
irrelevant and ratings of 3 and 4 were consid-
ered as relevant. The item level CVI was calcu-
lated as the number of experts rating an item as
relevant (i.e. 3 or 4) divided by the total number
of the experts. Generally, a limit of 0.78 is con-
sidered as the lower limit for content validity.
An overall CVI score for the instrument was cal-
culated as the proportion of the total number of
items deemed content valid. An acceptable
lower limit for the overall CVI is 0.8.28
Instrument testing
The final version of the developed instrument
was tested by six experts who evaluated 22 video
recorded in-hospital cardiopulmonary resuscita-
tions conducted by one resuscitation team in a
university hospital during the years 2013–2014.
These six experts participated also in the expert
panel. Cronbach’s a-values were calculated for
the evaluated TS (items 1–28) and NTS (items 29–
69). The a-values were calculated separately for
each expert due to dependencies in the data. They
are presented with mean values and confidence
intervals. The correlation of the TS, the NTS and
all items (1–69) with the overall performance
score given by the experts was calculated using
the Spearman’s correlation coefficient. SPSS ver-
sion 24 for Windows was used for the analysis.
Ethics
The study was approved by the Ethics Commit-
tee of the Hospital District of Southwest Finland
(statement number 14/2012, date of approval
19.6.2012).
Results
Item generation and instrument structure
The first version of the instrument was assem-
bled from the findings of the literature review,
the resuscitation guidelines, and the results
5
 L.-M. PELTONEN ET AL.
Table 2 Items and scales of instruments developed to assess resuscitation performance.
Authors (year)
10
Andersen et al. (2010)
Brett-Fleegler et al. (2008)11
Carmona Torres (2014)12
Cooper & Wakelam (1999)13
Cooper & Cant (2014),14
Cooper et al. (2014, 2016)15
Duff et al. (2013)16
Grant et al. (2012)17
Kim et al. (2006)18
Mancini & Kaye (1985)19
6 ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
No. of
Assessed dimensions items Rating alternatives
Initial therapy 22 ‘yes’
Continuous loops ‘no’
Information and supplementary therapy ‘not relevant’ option only for correction and
Spontaneous circulation technology dimensions
Correction
Maintain algorithm
Technology
Basic resuscitation 72 ‘yes’
Airway support ‘no’
Circulation and arrhythmia management
Leadership behaviour
Resuscitation guideline knowledge test 12 Four multiple-choice alternatives
Team task performance (basic ventilation, 9 ‘yes’
chest compression, advanced ventilation, ‘no’
defibrillation, intravenous access, ‘N/A’
medication, other treatment) Score 10–20 per task
Overall performance rating A total score is calculated as the sum of task
performance per the sum of applicable task
ratings
Leadership 12 0 = Never/hardly ever,
Teamwork 1 = Seldom
Task management 2 = About as often as not
Overall performance rating 3 = Often
4 = Always/Nearly always
Overall score from 1 to 10.
Algorithms and rhythm analysis 20 Each question had four possible answers
Team leadership including history taking,
classification and leadership behaviour
Basic life support algorithms
Advanced life support algorithms (such as
defibrillation, dose and definition of
bradycardia)
Leadership and communication skills 26 0 = Not performed/not observed
Knowledge and clinical skills 1 = Performed but ineffectively,
incompletely or inconsistently
2 = Performed adequately most of the time
3 = Performed well consistently
N/A = item not relevant to scenario
Leadership skills 6 Each item had specific scoring criteria
Problem solving A Likert-type scale ranging from 1 to 7,
Situational awareness where 1 represented a novice and 7 an expert
Resource utilisation
Communication skills
Overall crisis resource management rating
Basic skills 58 2 = Completed and timely
Airway and breathing 1 = Performed but incomplete
Circulation or untimely
Human factors 0 = Needed but not performed/performed
incorrectly
N/A = Not required for this scenario,
non-observable elements could be indicated.
Each item had specific scoring criteria
Acta Anaesthesiologica Scandinavica (2017)
 ALS INSTRUMENT DEVELOPMENT

Table 2 (Continued)

No. of
Authors (year) Assessed dimensions items Rating alternatives

Reid et al. (2012)20 Basic skills 94 2 Points = Completed and timely

Roh et al. (2012)21
Seaman et al. (1986)22
Turner et al. (2008)23
Walker et al. (2011)24
Zajano et al. (2014)25
Airway and breathing 1 Point = Performed but incomplete
Circulation or untimely
Human factors 0 Points = Needed but not
performed/performed incorrectly
N/A = Not required for this scenario
Non-observable elements
Resuscitation Self-efficacy: 17 A 5-point Likert-type scale
Recognition 1 = Least confident
Debriefing and recording 5 = Very confident
Responding and rescuing
Reporting
Defined tasks and subtasks based on the 16 ‘yes’
resuscitation guidelines ‘no’
Self-efficacy in resuscitation skills: 5 A visual analogue scale
Cardiac massage ‘not at all confident’ at the left and ‘extremely
Bag/mask ventilation confident’
Intra-osseous device at the right ends of the scale
Computer skills
Communication 48 Individual ratings from 0 to 6 on each task and a
Cooperation global behaviour score from 0 to 6 for each group
Coordination (anaesthetic, physician and nurse) under
Leadership each dimension.
Monitoring 0 = Team severely compromised
Decision making 1 = Team compromised
2 = Slight detriment to team
3 = Team neither enhanced or hindered
4 = Moderate enhancement to team
5 = High level of enhancement to team
6 = Highly effective in enhancing teamwork
Teamwork and communication 15 A 5-point Likert scale with rating options from
Task load strongly agree to strongly disagree

from the qualitative analysis of the interview
data. The instruments found in the literature
review assessed some aspects of TS (n = 3), NTS
(n = 5) or both (n = 7). However, none of these
were comprehensive, that is, all instruments
measured different things and consisted of dif-
ferent sets of items. There was, however, a sub-
stantial overlap in the assessed dimensions
between instruments. The included instruments,
scope of use and reported properties are pre-
sented in Table 1. Assessed dimensions, items
and scales are presented in Table 2. Individual
items were gathered from these instruments for
the instrument under development.
We interviewed 66 professionals, including
nurses (n = 35, 53%), orderlies (n = 8, 12%) and
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
physicians (n = 23, 35%). We identified 315
issues associated by the informants with the suc-
cess of ALS. Our analysis from the interview data
resulted in seven main themes. An example of
the analysis process is presented in Table 3 and
the main themes and subthemes in Table 4. The
main themes were used for grouping all items
into seven dimensions. The interview findings
were overlapping with the literature findings,
but also new items emerged. These were related
to patient privacy and work routines. One item
found in the literature did not emerge in the
interviews, namely providing feedback during
resuscitation to improve performance. New items
that emerged from the interviews were added to
the items extracted from the literature review
7
 L.-M. PELTONEN ET AL.
Table 3 Example of the thematic content analysis process.
Original expression
‘Cannulation and intubation was successful
at first attempt’
‘Using the protocol brings clarity. At least you
know what should be done. In the old days, you
had to figure out, what does this physician want?
Now we can plan better ahead and be more
systematic’
‘Nowadays, we have training, which is offered
regularly, this is important. It is very important
that all have a chance to attend the training’
‘We didn’t even check the rhythm as often as we
should have’
findings and the resuscitation guidelines. There-
after, the items were refined and overlapping
items were deleted. Resuscitation guidelines
were used to refine individual items.
This process resulted in a two-sided A4-sized
instrument with six dimensions: protocol adher-
ence (39 items), workload management and
decision-making (15 items), team behaviour (9
items), information management (9 items),
patient integrity and confidentiality (4 items),
and work routines (5 items). The protocol adher-
ence dimensions’ content was derived from exist-
ing resuscitation guidelines and findings from
the literature review. The 39 items in the protocol
adherence dimension were further grouped into
six subgroups. These were recognition of cardiac
arrest (6 items), choice of resuscitation algorithm
(3 items), chest compression quality (8 items),
ventilation quality (7 items), rhythm control and
defibrillation quality (9 items), and medication
and fluid therapy (5 items).
The response alternatives were developed
based on the findings from the literature review.
It was a semantic differential type of scale with
four response options and a fifth option for ‘not
applicable’ situations. The four-scale options
were ‘fully completed’, ‘partly completed’, ‘not
needed’ and ‘needed but not completed’.
The content validity of the first version of
the instrument
The developed instrument was evaluated by 20
experts. The mean working years of the experts
8 ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
Meaning unit Subtheme Main theme
The success of the technical procedures Technical Adherence
is important know-how to guidelines
(positive association)
Training has improved protocol adherence Theoretical
and made functioning more systematic knowledge
(positive association)
Enabling professionals to attend in Professional
training is important development
(positive association)
Protocol was not followed Protocol
(negative association) commitment
was 15.7 (IQR 9–20) years. The demographics of
the expert panel members are presented in
Table 5.
Changes were made to the instrument based
on the expert panel results. The CVI for all the
84 items was 0.96. Three individual items had a
value below the recommended 0.78. These were
as follows: ‘Did the leader let the team members
know what was expected of them?’ (CVI 0.61),
‘Were general aseptic recommendations fol-
lowed?’ (CVI 0.58) and ‘Were general hand
hygiene recommendations followed?’ (CVI
0.53). These items were removed from the
instrument. Based on the open comments from
the experts, the response alternatives were clari-
fied, 13 individual items were left unaltered,
three new items were added, 22 items were
overlapping and modified into 8 items, and
minor changes (such as clarity, wording or
placement of item) were made to 47 items.
Examples of minor changes were as follows:
‘was the situation documented?’ was clarified
into ‘Was the cardiac resuscitation process docu-
mented?’ and ‘Was the compressors hand posi-
tion correct?’ was modified into ‘Was the
compression technique correct?’
The final version of the instrument
The final version of the developed instrument
for the evaluation of ALS performance is
presented in Table 6. This final version had
69 items, which were divided into seven dimen-
sions. These were adherence to guidelines (28
Acta Anaesthesiologica Scandinavica (2017)
 ALS INSTRUMENT DEVELOPMENT

Table 4 Factors associated with the success of advanced life support by resuscitation team professionals.

Main theme Subtheme

Adherence to Technical know-how An introduction to team functions and equipment enabled

guidelines smoother functioning.
Theoretical knowledge Team members expected the resuscitation to follow guidelines
throughout the process.
Professional development Professionals were motivated to attend in training and to improve their
knowledge and skills, but finding time to attend was considered more
difficult due to clinical work urgencies.
Protocol commitment The resuscitation was seen as a process starting before the team was
on scene. The initiating activities of the professionals calling the
resuscitation team, such as immediately started chest compressions,
were considered important ingredients of ALS performance. The
reported activities that diverged from protocol related to time,
medication and securing the airway.
Clinical Patient-care-related Care-related decision-making included decisions about when to start
decision-making decision-making CPR and when not to. These decisions were sometimes difficult to
make based on the knowledge at hand.
Clinical decision-making Decision-making responsibilities regarding medical care were not always
responsibilities clear and the decision-making could become complicated when several
physicians were on scene: for example, when three physicians were on
scene, none took charge of the situation. However, shared decision-
making could be considered a strength when the collaboration
functioned well.
Workload Responsibilities of Team members appreciated clear and well-defined roles and
management team members responsibilities that they could stick to. Specific responsibilities were
continuous compressions, securing the airway, medication
administration and documentation.
Leadership Professionals reported a difference between leading the resuscitation
protocol and leading the situation overall. The overall leadership
meant that someone took responsibility for continuously monitoring the
patient’s condition, constructing and updating an action plan,
communicating this plan to all team members and delegating tasks.
Delegating the resuscitation protocol leadership to a nurse enabled the
physician to spend more time on the underlying reasons for the event
and providing leadership for the overall situation.
Activities of other Some responsibilities, such as documentation, could be delegated to
professionals for example ward nurses.
Team structure Number of The number of professionals in the team varied, but there was a
and behaviour professionals involved minimum of two registered nurses and one physician, usually an
anaesthetist. This number was thought to be sufficient in normal
resuscitations. Too many professionals could cause confusion.
Characteristics of The personal characteristics of team members were associated with team
team members behaviour. Things could go smoothly or less smoothly depending on
the members involved. Experienced professionals were appreciated.
Commitment to The team members’ commitment to teamwork involved how they carried
teamwork out their assigned duties and assisted others.
Atmosphere A calm atmosphere was preferred and a resuscitation could be
during the CPR considered as successful by the professionals, even if the patient died,
if the resuscitation had advanced calmly.
Information Calling the During the call, the team expected information about the location of the
management resuscitation team cardiac arrest, but also other information, such as a possible need for
infectious precautions, so that they were appropriately prepared on
arrival.

Acta Anaesthesiologica Scandinavica (2017)

ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd 9
 L.-M. PELTONEN ET AL.

Table 4 (Continued)

Main theme Subtheme

Communication during CPR Communication with other than team members was described as
sometimes being difficult. Team members preferred that the leader of
the resuscitation protocol continuously talked aloud about the
upcoming events of the protocol beforehand. This enabled them to
prepare and, for example, to facilitate a smoother change between
persons conducting chest compressions. Communication further
included giving clear orders, using closed loop communication and
using name tags. Keeping the whole team up to date was
considered important.
Patient information A major concern was the difficulty to access relevant patient information
at the scene of the resuscitation. Ward nurses did not always have
sufficient information about the patient and getting this information was
difficult. One professional suggested adding a laptop to the
resuscitation trolley for better access to important information.
Patient privacy Patient privacy Patient privacy refers to the ability of a patient to seclude him or herself
and laymen or information about him or herself from others: for example,
protecting the body or personal information of a patient
from bystanders.
Consideration of laymen The experiences of laymen needed to be acknowledged, and a need to
reflect on the event should be recognised, as some laymen might need
support after witnessing a resuscitation.
Work routines The team’s arrival at A guiding person to meet the resuscitation team was associated with
the scene of resuscitation a smoother arrival.
Consideration of The consideration of situation-specific issues related to the environment,
situation-specific issues such as the angiotherapy unit, the magnetic resonance imaging (MRI)
room, a toilet and an isolation room. A need for clear instructions
regarding how to function under special circumstances, such as during
MRI or angiotherapy, was reported. A further report focused on
ergonomic challenges and the optimal use of restricted physical
environments.
Equipment Immediate and easy access to necessary equipment was vital. Equipment
referred to the resuscitation trolley with its standardised content and
supplementary equipment would be requested and fetched when
necessary: for example, a videolaryngoscope during a
difficult intubation.
Transfer into The transfer into post-resuscitation care was supported by the fact that
post-resuscitation care the resuscitation team physician often was responsible for the post-
resuscitation care in the hospital. A major issue with the transfer into
post-resuscitation care was the lack of available beds.

The seven main themes and their subthemes, which were identified from the interview data with resuscitation team professionals, are
presented.

items), clinical decision-making (5 items), chest compression quality (9 items), ventilation

workload management (12 items), team beha-
viour (8 items), information management (6
items), patient integrity and consideration of
laymen (4 items), and work routines (6 items).
The adherence to guidelines dimension had five
subgroups. These were recognition of the need
for cardiopulmonary resuscitation (5 items),
10
quality (4 items), rhythm control and defibrilla-
tion quality (6 items), and medication and fluid
therapy (4 items). The items may be rated on a
scale ranging from +2 to 2. The rating alterna-
tives are detailed in Table 7. Thirteen of the
items in the instrument may only be answered
as +2, indicating ‘yes’, 2 indicating ‘no’ or 0
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
 ALS INSTRUMENT DEVELOPMENT

example of the instruction for one item is shown

Table 5 Demographics of the expert panel.
in Table 8.
Demographics of the expert panel members n (%)

Profession Validity and reliability of the developed

Nurse 14 (70) instrument
Physician 6 (30)
Specialisation The experts who tested the developed instru-
Intensive care 5 (25) ment included nurses (n = 2) and physicians
Anaesthesiology 1 (5) (n = 4). Their mean working experience was
Anaesthesiology and intensive care 3 (15) 14.9 (IQR 10–20) years and their resuscitation
Operative setting and anaesthesiology 2 (10)
experience included resuscitation situations
Anaesthesiology, intensive care and prehospital care 2 (10)
Emergency medical services 7 (35)
(< 10 n = 1, 10 49 n = 2, 50 99 n = 1, > 100
Working experience n = 3), resuscitation simulations (10 49 n = 5,
Hospital 5 (25) > 100 n = 1) and resuscitation simulation teach-
Emergency medical services 2 (10) ing (< 10 n = 3, 10 49 n = 1, > 100 n = 2). The
Hospital and emergency medical services 8 (40) Cronbach’s a values were 0.83 for the TS (95%
Educational institution and hospital 2 (10) C.I. 0.78–0.88) and 0.82 for the NTS (95% C.I.
Educational institution and 1 (5)
0.71–0.93). Strong positive correlations between
emergency medical services
Educational institution, hospital 2 (10)
the experts’ evaluation of the overall perfor-
and emergency medical services mance were observed with the TS (rs = 0.67,
Resuscitation situations P < 0.001), the NTS (rs = 0.69, P < 0.001) and all
1 < 10 1 (5) items (rs = 0.73, P < 0.001).
2 = 10 49 8 (40)
3 = 50 99 4 (20)
4 > 100 7 (35) Discussion
Resuscitation simulations
1 < 10 8 (40)
A resuscitation team’s performance is measured
2 = 10 49 11 (55) not only by the skills of an individual but on
3 = 50 99 0 (0) the performance of the whole team and a variety
4 > 100 1 (5) of external factors. Existing validated instru-
Resuscitation simulation teaching ments focus on some aspects of TS, NTS or both,
1 < 10 7 (35) but all current instruments lack important
2 = 10 49 9 (45)
aspect needed for a comprehensive and detailed
3 = 50 99 1 (5)
4 > 100 3 (15)
assessment of the overall ALS performance of
the whole resuscitation team. ALS is guided by
international guidelines; therefore, a single
instrument could possibly be used to evaluate
indicating ‘not possible to evaluate’. The ALS performance regardless of the setting.
extremes (+2 and 2) were chosen to increase Although, guidelines alone do not provide all
the distinction between the amount of success items needed. According to our findings, the
in performance. In addition, a final item was overall performance of ALS should be assessed
added at the end of the instrument for the total from a much broader view. A comprehensive
assessment score of the performance. This had a overall assessment should acknowledge adher-
scale from 0 to 10 (0 = poor performance, ence to guidelines, clinical decision-making,
10 = excellent performance). This overall score workload management, team structure and
is important for assessing the validity of the behaviour, information management, patient
instrument. privacy and laymen, and work routines.
We defined detailed descriptions of the evalu- The instrument in this study was developed
ation criteria for each item in the instrument. for a comprehensive assessment of the overall
These were derived from resuscitation guide- ALS performance of the whole resuscitation
lines. This 13-page guideline for using the team in education and research, hence reducing
instrument is available in Appendix S1. An the need to use numerous different instruments.
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd 11
 L.-M. PELTONEN ET AL.

Table 6 Developed instrument for the evaluation of advanced life support performance.
INSTRUMENT FOR THE EVALUATION OF ADVANCED LIFE SUPPORT PERFORMANCE
Peltonen L-M, Tommila M & Salanter€a S (2015)

Date: ____/____ ________________________________________ Assessment no.: ______ Rating alternatives

Purpose: Education ______ OR Research ______ +2 +1 0 1 2
Evaluation ______ OR Reflection/learning ______
Situation: Simulation ______ OR Clinical setting ______
Team: profession _____________ (no. ___), profession _____________ (no. ___),
profession _____________ (no. ___), profession _____________ (no. ___)

1. ADHERENCE TO GUIDELINES
A. Recognition of the need for cardiopulmonary resuscitation
1 Was the need for cardiopulmonary resuscitation immediately recognised? +2 X 0 X 2
2 Was help called for immediately? +2 X 0 X 2
3 Were chest compressions initiated immediately? +2 X 0 X 2
4 Was the airway opened and breathing assessed? +2 X 0 X 2
5 Was the patient lifted on a firm surface or was a backboard placed when necessary? +2 X 0 X 2
B. Chest compression quality
6 Was the compression technique correct? +2 +1 0 1 2
7 Was the compression rate 100–120 per min.? +2 +1 0 1 2
8 Was the compression-ventilation rate correct? +2 +1 0 1 2
9 Was the compression depth correct? +2 +1 0 1 2
10 Was there complete chest recoil between compressions? +2 +1 0 1 2
11 Was the number of interruptions in the chest compressions minimised? +2 +1 0 1 2
12 Was the length of the interruptions in the chest compressions < 10 s.? +2 +1 0 1 2
13 Was the CPR quality evaluated? +2 +1 0 1 2
14 Was the compressor rotation adequate? +2 +1 0 1 2
C. Ventilation quality
15 Was 100% oxygen administered as soon as possible? +2 +1 0 1 2
16 Was the airway ensured using appropriate equipment as soon as possible? +2 +1 0 1 2
17 Was placement of an advanced airway ensured? +2 +1 0 1 2
18 Did ventilation actualise according to guidelines? +2 +1 0 1 2
D. Rhythm control and defibrillation quality
19 Were the pads placed as soon as the defibrillator was on site? +2 X 0 X 2
20 Were the pads placed correctly on the patient’s chest? +2 X 0 X 2
21 Was the rhythm (VT/VF/ASY/PEA) correctly identified? +2 +1 0 1 2
22 Did defibrillation actualise according to guidelines if the rhythm was VF/VT +2 +1 0 1 2
OR were compressions immediately continued if the rhythm was PEA/ASY?
23 Was rhythm analysis performed with 2 min. intervals? +2 +1 0 1 2
24 Was the resuscitation algorithm followed? +2 +1 0 1 2
E. Medication and fluid therapy
25 Was a parenteral medication route inserted as soon as possible? +2 X 0 X 2
26 Was medication use appropriate? +2 +1 0 1 2
27 Was a suitable amount of fluid given after medication to facilitate the drug flow? +2 +1 0 1 2
28 Were medicines administered as ordered? +2 +1 0 1 2
2. CLINICAL DECISION-MAKING
29 Were possible causes for the cardiac arrest analysed? +2 +1 0 1 2
30 Were reversible causes for the cardiac arrest treated? +2 X 0 X 2
31 Were tasks prioritised appropriately? +2 +1 0 1 2
32 Was vital organ support optimised if ROSC was attained? +2 +1 0 1 2
33 Was an appropriate care plan made for the patient if ROSC was attained? +2 +1 0 1 2

Acta Anaesthesiologica Scandinavica (2017)

12 ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
 ALS INSTRUMENT DEVELOPMENT

Table 6 (Continued)

INSTRUMENT FOR THE EVALUATION OF ADVANCED LIFE SUPPORT PERFORMANCE

Peltonen L-M, Tommila M & Salanter€
a S (2015)

Rating alternatives
+2 +1 0 1 2
3. WORKLOAD MANAGEMENT
34 Were there a suitable number of professionals present? +2 +1 0 1 2
35 Was it clear who led the team? +2 +1 0 1 2
36 Were all team members’ roles clear? +2 +1 0 1 2
37 Were all roles appropriate for the team members? +2 +1 0 1 2
38 Did the leader continuously outline a plan of action? +2 +1 0 1 2
39 Did a team member continuously maintain the overview of the situation? +2 +1 0 1 2
40 Did a team member continuously monitor the patient’s condition? +2 +1 0 1 2
41 Did a team member continuously give corrective feedback about the quality of CPR? +2 +1 0 1 2
42 Did a team member continuously follow -up the adherence to the resuscitation algorithm? +2 +1 0 1 2
43 Did a team member continuously inform the others when it was time for an +2 +1 0 1 2
intervention according to the resuscitation algorithm?
44 Did a team member continuously inform others about changes in the patient’s condition? +2 +1 0 1 2
45 Did the team members carry out their responsibilities? +2 +1 0 1 2
4. TEAM BEHAVIOUR
46 Did the team members ask for help when needed? +2 +1 0 1 2
47 Did the team members assist each other when needed? +2 +1 0 1 2
48 Was the situation handled calmly? +2 +1 0 1 2
49 Did the team members behave professionally towards each other? +2 +1 0 1 2
50 Did the team members behave professionally towards the patient? +2 +1 0 1 2
51 Did the team members correct each other when necessary? +2 +1 0 1 2
52 Did the team members inform others about their observations and insights? +2 +1 0 1 2
53 Was the leader open to suggestions from other team members? +2 +1 0 1 2
5. INFORMATION MANAGEMENT
54 Was communication between the team members clear? +2 +1 0 1 2
55 Was ‘closed loop communication’ used? +2 +1 0 1 2
56 Was the number of chest compressions counted aloud? +2 +1 0 1 2
57 Was relevant patient information shared effectively between professionals? +2 +1 0 1 2
58 Was additional patient information made available? +2 X 0 X 2
59 Was the cardiac resuscitation process documented? +2 +1 0 1 2
6. PATIENT INTEGRITY AND CONSIDERATION OF LAYMEN
60 Was patient integrity protected? +2 +1 0 1 2
61 Was patient information handled discretely? +2 +1 0 1 2
62 Were laymen guided away from the resuscitation site when necessary? +2 X 0 X 2
63 Were laymen given support when necessary? +2 X 0 X 2
7. WORK ROUTINES
64 Did the team members protect themselves adequately? +2 +1 0 1 2
65 Did the team find the site of the cardiac resuscitation quickly and easily? +2 +1 0 1 2
66 Did the team members have the right equipment? +2 X 0 X 2
67 Did the team members know how to use the equipment? +2 +1 0 1 2
68 Were the team members able to take into consideration the physical resources and restraints of the site? +2 +1 0 1 2
69 Were the team members working ergonomically? +2 +1 0 1 2

THE OVERALL PERFORMANCE SCORE (0 = POOR PERFORMANCE, 10 = EXCELLENT PERFORMANCE): _______

When compared with existing validated instru- patient integrity and work routines. These novel
ments, the developed instrument includes addi- aspects support a more detailed assessment of
tional items, which are not seen in earlier the overall ALS performance of the team when
validated instruments. These are related to compared with other instruments.
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd 13
 L.-M. PELTONEN ET AL.
Table 7 The rating alternatives.
Score Meaning
+2
 Yes
 Yes, always
 Correct actions were taken in all the situations
 Correct actions were taken all the time
 Completely correct actions were taken
 Completely correct
+1
 Yes, most of the time
 Correct actions were taken in most of the situations
 Correct actions were taken most of the time
 Mostly correct
0 This option should only be used when the question
cannot be evaluated due to technical failures or to
other audial or visual obstacles or hindrances. Except
for questions 32 and 33.
1 thumbs while the hands are around the
 No, most of the time
 Incorrect actions were taken in most of the situations
 Incorrect actions were taken most of the time
 Mostly incorrect
2 hand position on the patient’s
 No
 No, never
 Incorrect actions were taken in all the situations
 Incorrect actions were taken all the time
 Completely incorrect actions were taken
 Completely incorrect
The literature review, the resuscitation guide-
lines and the interviews provided items for the
developed instrument. These were mainly over-
lapping. Patient integrity and work routines
related issues did not appear in the literature
review, while the provision of feedback was
lacking from the interview data. The reason for
the lack of patient integrity and work routine
issues from the literature may be the difficulty
to operationalise and measure the concepts.
However, measuring the performance of ALS
in sustaining life is prioritised. Providing feed-
back to team members about the resuscitation
performance has been shown to improve team
performance.30,31 One possible reason for the
interviews lacking this aspect is that resuscita-
tion feedback nowadays often is received
straight from the defibrillator. Another might be
14
Table 8 Example of detailed evaluation criteria for an item in the
guideline for using the instrument.
Chest compression quality
Was the compression Adults: compressions are performed on
technique correct? the middle of the sternum with the
palm of one hand at the base and the
other hand on it with arms straight and
shoulders perpendicularly above the
patient’s chest. The fingers are kept
interlocked and bent off the chest to
prevent ribs from breaking.
Children: compressions are performed
at the lower part of the sternum.
In compressions of a 1- year old child
to a teenager one or two palms are
used. Only two fingers are used for a
child less than 1-year old. When there is
more than one professional on site,
compressions are performed with the
child’s chest.
+2 All compressors had a correct hand
position on the patient’s chest
and the right compression angle.
+1 Most compressors had a correct
chest and the right compression
angle.
0 Not possible to evaluate.
1 Most compressors had an incorrect
hand position on the patient’s
chest and the wrong compression
angle.
2 All compressors had an incorrect
hand position on the patient’s
chest and the wrong compression
angle.
that there is a lack of such a feedback culture in
the organisation.
The current literature supports all the aspects
identified in our analysis as being associated
with ALS performance. Adherence to guidelines
is the most obvious and probably most often
evaluated aspect of these,7 as a correlation
between knowledge, skills and resuscitation
performance is clearly evident.10–12,16,17,19–23,32,33
Existing literature and developed instruments
also support the relevance of clinical decision-
making,10,11,13–15,17–21,24,34 workload manage-
ment,10,11,13–18,20,21,24,25,35,36 the team structure
and behaviour,10,11,13–15,20,21,24,25,37,38 informa-
tion management,10,11,14–18,20,21,24,25,39 patient
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd

integrity and consideration of laymen,40,41 and
work routines.10,11,20,42,43
Due to the large number of items, a real-time
assessment of ALS with the developed instru-
ment is impractical. Therefore, video recording
is recommended to support the assessment for
reliable results of performance both in authentic
situations and in simulations. Based on further
research, including assessment of the relevance
of the items and prioritisation of the items, a
shorter version of the instrument could be
developed for real-time use. Further research
would also provide information about how to
best compute performance scoring. Research is
also needed to further develop the anchors in
the guidelines and to assess the psychometric
properties of the instrument.
Different research purposes, local conditions
or educational needs may benefit from addi-
tional items. For example, sustaining life is the
most important aim of ALS, but paying atten-
tion to other issues such as privacy and hygiene
may improve the given care. Following hygiene
and aseptic techniques is often possible even
when sustaining life is prioritised. The items
regarding hygiene and aseptic techniques were
removed from the instrument based on the
experts’ assessment. These items may, however,
be included in studies, which investigate possi-
ble associations between hygiene and aseptic
techniques with long-term resuscitation out-
comes.
The strong positive correlation between the
experts’ overall assessment score and the instru-
ment items indicate that the developed instru-
ment measures what it is supposed to measure.
The good Cronbach’s a values indicate good
internal consistency. However, more testing is
needed to verify the findings and to assess the
psychometric properties of the instrument in
more detail. The credibility of the findings was
increased by choosing interview informants and
experts of different gender, age, profession and
with different work experience. This variety
may shed light on the research question from a
variety of perspectives.29 Although one of the
authors was in charge of the qualitative analy-
sis, the whole research team participated in
assessing the results of the analysis. The trans-
ferability of the results is limited to the sample,
that is, we only interviewed professionals from
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
ALS INSTRUMENT DEVELOPMENT
one university hospital. Research in other set-
tings is needed to further validate the findings.
Conclusions
A comprehensive overall ALS assessment should
acknowledge adherence to guidelines, clinical
decision-making, division of labour, team struc-
ture and behaviour, information management,
patient privacy and laymen, and work routines.
We developed an instrument for the detailed eval-
uation of ALS performance based on existing liter-
ature, resuscitation guidelines and interviews
conducted with professionals working in a
university hospitals resuscitation team. The devel-
oped instrument may be useful in education and
research for a detailed assessment of an overall
ALS performance. However, it should be empha-
sised that the instrument is still under develop-
ment and empirical testing is needed to determine
its psychometric properties and to assess its
usability. Therefore, using the current version of
the instrument should be done with consideration.
Acknowledgement
This study received support from the Finnish
Medical Society Duodecim, Turunmaa Chapter.
The authors state no conflict of interest.
References
1. Broomfield R. A quasi experimental research to
investigate the retention of basic cardiopulmonary
resuscitation skills and knowledge by qualified
nurses following a course in professional
development. J Adv Nurs 1996; 23: 1016–23.
2. Howard-Quijano K, Stiegler M, Huang Y, Canales
C, Steadman R. Anesthesiology residents’
performance of pediatric resuscitation during a
simulated hyperkalemic cardiac arrest.
Anesthesiology 2010; 112: 993–7.
3. Passali C, Pantazopoulos I, Dontas I, Patsaki A,
Barouxis D, Troupis G, Xanthos T. Evaluation of
nurses’ and doctors’ knowledge of basic &
advanced life support resuscitation guidelines.
Nurse Educ Pract 2011; 11: 365–9.
4. Allen J, Currey J, Considine J. Annual
resuscitation competency assessments: a review of
the evidence. Aust Crit Care 2013; 26: 12–17.
5. Williams N. Advanced life support training and
assessment: a literature review. Australas Emerg
Nurs J 2011; 14: 240–5.
15
 L.-M. PELTONEN ET AL.
6. Preston J, Currey J, Eastwood G. Assessing
advanced life support (ALS) competence: Victorian
practices. Aust Crit Care 2009; 22: 164–71.
7. M€akinen M, Niemi-Murola L, M€ akel€
a M, Castren
M. Methods of assessing cardiopulmonary
resuscitation skills: a systematic review. Eur J
Emerg Med 2007; 14: 108–14.
8. Jepsen R, Østergaard D, Dieckmann P.
Development of instruments for assessment of
individuals’ and teams’ non-technical skills in
healthcare: a critical review. Cogn Technol Work
2015; 17: 63–77.
9. Perkins G, Hulme J, Tweed M. Variability in the
assessment of advanced life support skills.
Resuscitation 2001; 50: 281–6.
10. Andersen P, Jensen M, Lippert A, Østergaard D,
Klausen T. Development of a formative assessment
tool for measurement of performance in multi-
professional resuscitation teams. Resuscitation
2010; 81: 703–11.
11. Brett-Fleegler M, Vinci R, Weiner D, Harris S, Shih
M, Kleinman M. A simulator-based tool that
assesses pediatric resident resuscitation
competency. Pediatrics 2008; 121: e597–603.
12. Carmona Torres J. Questionnaire on nurses’
knowledge regarding the current cardiopulmonary
resuscitation recommendations 2010. Enferm Clin
2014; 24: 308–10.
13. Cooper S, Wakelam A. Leadership of resuscitation
teams: “Lighthouse Leadership”. Resuscitation
1999; 42: 27–45.
14. Cooper S, Cant R. Measuring non-technical skills
of medical emergency teams: an update on the
validity and reliability of the Team Emergency
Assessment Measure (TEAM). Resuscitation 2014;
85: 31–3.
15. Cooper S, Cant R, Connell C, Sims L, Porter J,
Symmons M, Nestel D, Liaw S. Measuring
teamwork performance: validity testing of the Team
Emergency Assessment Measure (TEAM) with
clinical resuscitation teams. Resuscitation 2016;
101: 97–101.
16. Duff J, Cheng A, Bahry L, Hopkins J, Richard M,
Schexnayder S, Carbonaro M. EXPRESS
investigators. Development and validation of a
multiple choice examination assessing cognitive and
behavioural knowledge of pediatric resuscitation: a
report from the EXPRESS pediatric research
collaborative. Resuscitation 2013; 84: 365–8.
17. Grant E, Grant V, Bhanji F, Duff J, Cheng A,
Lockyer J. The development and assessment of an
evaluation tool for pediatric resident competence in
16 ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
leading simulated pediatric resuscitations.
Resuscitation 2012; 83: 887–93.
18. Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J.
A pilot study using high-fidelity simulation to
formally evaluate performance in the resuscitation
of critically ill patients: the University of Ottawa
Critical Care Medicine, High-Fidelity Simulation,
and Crisis Resource Management I Study. Crit Care
Med 2006; 34: 2167–74.
19. Mancini M, Kaye W. Development and validation
of an objective criterion-referenced tool for
evaluation of advanced cardiac life support
performance. Heart Lung 1985; 14: 299–300.
20. Reid J, Stone K, Brown J, Caglar D, Kobayashi A,
Lewis-Newby M, Partridge R, Seidel K, Quan L.
The Simulation Team Assessment Tool (STAT):
development, reliability and validation.
Resuscitation 2012; 83: 879–86.
21. Roh Y, Issenberg S, Chung H, Kim S. Development
and psychometric evaluation of the Resuscitation
Self-efficacy Scale for nurses. J Korean Acad Nurs
2012; 42: 1079–86.
22. Seaman J, Greene B, Watson-Perczel M. A
behavioral system for assessing and training
cardiopulmonary resuscitation skills among
emergency medical technicians. J Appl Behav Anal
1986; 19: 123–35.
23. Turner N, van de Leemput A, Draaisma J, Oosterveld
P, ten Cate O. Validity of the visual analogue scale as
an instrument to measure self-efficacy in
resuscitation skills. Med Educ 2008; 42: 503–11.
24. Walker S, Brett S, McKay A, Lambden S, Vincent
C, Sevdalis N. Observational skill-based clinical
assessment tool for resuscitation (OSCAR):
development and validation. Resuscitation 2011;
82: 835–44.
25. Zajano E, Brown L, Steele D, Baird J, Overly F,
Duffy S. Development of a survey of teamwork and
task load among medical providers: a measure of
provider perceptions of teamwork when caring for
critical pediatric patients. Pediatr Emerg Care 2014;
30: 157–60.
26. Flanagan JThe critical incident technique. Psychol
Bull 1954; 51: 327–58.
27. Graneheim U, Lundman B. Qualitative content
analysis in nursing research: concepts, procedures
and measures to achieve trustworthiness. Nurse
Educ Today 2004; 24: 105–12.
28. Polit D, Beck C. The content validity index: are you
sure you know what’s being reported? Critique
and recommendations Res Nurs Health 2006; 29:
489–97.
Acta Anaesthesiologica Scandinavica (2017)

29. DeVon H, Block M, Moyle-Wright P, Ernst D,
Hayden S, Lazzara D, Savoy S, Kostas-Polston E. A
psychometric toolbox for testing validity and
reliability. J Nurs Scholarsh 2007; 39: 155–64.
30. Yeung J, Meeks R, Edelson D, Gao F, Soar J,
Perkins G. The use of CPR feedback/prompt
devices during training and CPR performance: a
systematic review. Resuscitation 2009; 80: 743–51.
31. Edelson D, Litzinger B, Arora V, Walsh D, Kim S,
Lauderdale D, Vanden Hoek T, Becker L, Abella B.
Improving in-hospital cardiac arrest process and
outcomes with performance debriefing. Arch Intern
Med 2008; 168: 1063–9.
32. Burkhardt J, Glick J, Terndrup T. Effect of prior
cardiopulmonary resuscitation knowledge on
compression performance by hospital providers.
West J Emerg Med 2014; 15: 404–8.
33. Brown T, Dias J, Saini D, Shah R, Cofield S,
Terndrup T, Kaslow R, Waterbor J. Relationship
between knowledge of cardiopulmonary
resuscitation guidelines and performance.
Resuscitation 2006; 69: 253–61.
34. Mills P, DeRosier J, Neily J, McKnight S, Weeks
W, Weeks J. A cognitive aid for cardiac arrest: you
can’t use it if you don’t know about it. Jt Comm J
Qual Patient Saf 2004; 30: 488–96.
35. Sayee N, McCluskey D. Factors influencing
performance of cardiopulmonary resuscitation
(CPR) by Foundation Year 1 hospital doctors.
Ulster Med J 2012; 81: 14–18.
36. Shetty P, Cohen T, Patel B, Patel V. The cognitive
basis of effective team performance: features of
failure and success in simulated cardiac
resuscitation. AMIA Annu Symp Proc 2009; 2009:
599–603.
37. Hunziker S, Johansson A, Tschan F, Semmer N,
Howell M, Marsch S. Teamwork and leadership in
Acta Anaesthesiologica Scandinavica (2017)
ª 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd
ALS INSTRUMENT DEVELOPMENT
cardiopulmonary resuscitation. J Am Coll Cardiol
2011; 57: 2381–8.
38. Krage R, Tjon Soei Len L, Schober P, Kolenbrander
M, vanGroeningen D, Loer S, Wagner C, Zwaan L.
Does individual experience affect performance
during cardiopulmonary resuscitation with
additional external distractors? Anaesthesia
2014;69:983–9.
39. You J, Park S, Park I. The development of effective
communication instructions may improve the
quality of CPR performance. Resuscitation 2011; 82:
237.
40. Walker W. Witnessed resuscitation: a concept
analysis. Int J Nurs Stud 2006; 43: 377–87.
41. Møller T, Hansen C, Fjordholt M, Pedersen B,
Østergaard D, Lippert F. Debriefing bystanders of
out-of-hospital cardiac arrest is valuable.
Resuscitation 2014; 85: 1504–11.
42. Jones C, Thorne C, Colter P, Macrae A, Brown G,
Hulme J. Rescuers may vary their side of approach
to a casualty without impact on cardiopulmonary
resuscitation performance. Emerg Med J 2013; 30:
74–5.
43. Cho J, Oh J, Park Y, Park I, Chung S. Effects of
bed height on the performance of chest
compressions. Emerg Med J 2009; 26: 807–10.
Supporting Information
Additional Supporting Information may be
found in the online version of this article at the
publisher’s web-site:
Appendix S1. Instructions for the use of the
instrument for the evaluation of advanced life
support performance
17