Volume 46 & Number 2 & April 2014 79

A Pediatric FOUR Score Coma Scale:

Interrater Reliability and Predictive Validity
Brianna L. Czaikowski, Hong Liang, C. Todd Stewart

ABSTRACT
The Full Outline of UnResponsiveness (FOUR) Score is a coma scale that consists of four components
(eye and motor response, brainstem reflexes, and respiration). It was originally validated among the adult
population and recently in a pediatric population. To enhance clinical assessment of pediatric intensive
care unit patients, including those intubated and/or sedated, at our children’s hospital, we modified the
FOUR Score Scale for this population. This modified scale would provide many of the same advantages
as the original, such as interrater reliability, simplicity, and elimination of the verbal component that is
not compatible with the Glasgow Coma Scale (GCS), creating a more valuable neurological assessment tool for
the nursing community. Our goal was to potentially provide greater information than the formally used GCS
when assessing critically ill, neurologically impaired patients, including those sedated and/or intubated.
Experienced pediatric intensive care unit nurses were trained as ‘‘expert raters.’’ Two different nurses assessed
each subject using the Pediatric FOUR Score Scale (PFSS), GCS, and Richmond Agitation Sedation Scale at
three different time points. Data were compared with the Pediatric Cerebral Performance Category (PCPC)
assessed by another nurse. Our hypothesis was that the PFSS and PCPC should highly correlate and
the GCS and PCPC should correlate lower. Study results show that the PFSS is excellent for interrater reliability
for trained nurseYrater pairs and prediction of poor outcome and in-hospital mortality, under various situations,
but there were no statistically significant differences between the PFSS and the GCS. However, the PFSS
does have the potential to provide greater neurological assessment in the intubated and/or sedated patient
based on the outcomes of our study.

Keywords: coma scale/standards, consciousness/nursing, pediatric nursing, predictive value of tests,

rater/observer variation, severity of illness index

D
espite advances in technology and state-of-
the-art monitoring devices, a comprehensive
clinical assessment is the means to recognize
subtle changes in a patient’s neurological status, whether
conscious or not. It also must allow for accurate and
consistent communication of these changes by nurses
and other healthcare providers. To assist in this assess-
ment and thereby provide quality patient care, several
validated neurological assessment tools have been de-
veloped over the past 50 years.
Questions or comments about this article may be directed to
Brianna L. Czaikowski, RN, at czaikowski.brianna@ministryhealth.org.
She is a Registered Nurse in the Pediatric Intensive Care Unit, Ministry
St. Joseph’s Hospital, Marshfield, WI.
Hong Liang, PhD, is a Biostatistician in the Biomedical Informatics
Research Center, Marshfield Clinic Research Foundation,
Marshfield, WI.
C. Todd Stewart, MD, was the Vice President and a Pediatric
Intensivist at Marshfield Clinic/Ministry St. Joseph’s Hospital,
Marshfield, WI.
This study was funded by the Marshfield Clinic’s physician
research funds and the Marshfield Clinic Research Foundation’s
disease specific funds.
The authors declare no conflicts of interest.
Copyright B 2014 American Association of Neuroscience Nurses
DOI: 10.1097/JNN.0000000000000041
In 1966, one of the first neurological assessment
tools developed was the Ommaya ‘‘vital sign’’ card
(Cohen, 2009; Ommaya, 1966). In 1974, it was ex-
panded by Teasdale and Jennett (1974) into the Coma
Index, then became known as the Glasgow Coma
Scale (GCS). The GCS is used internationally, in both
prehospital and hospital settings, to predict morbidity,
mortality, and long-term outcomes in acute neurosci-
ence patients. Over the years, many efforts have been
made to improve the GCS, yet these tools have rarely
been accepted into practice (Cohen, 2009).
The Full Outline of UnResponsiveness (FOUR)
Score Coma Scale was developed in 2005 by research-
ers at the Mayo Clinic, more specifically, Dr. Eelco
Wijdicks, to enhance the clinical assessment of patients
by improving communication among healthcare per-
sonnel (Wijdicks, Bamlet, Maramattom, Manno, &
McClelland, 2005). This original FOUR Score Scale
was aimed toward adult patients who were not sedated;
however, further studies showed its simplicity, ability
to be used outside of the neurological intensive care
unit (ICU), and ability to assess patients receiving mild
sedation.
The FOUR Score Scale was validated in the adult
population and proven to be an effective alternative to
the GCS (which was tested and validated in only adult

Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
80 Journal of Neuroscience Nursing
trauma, mainly head injury, patients); however, it was
not validated in a pediatric population until 2009. The
Cohen (2009) study focused on validating the FOUR
Score Scale in pediatric patients, with the exclusion
of patients receiving sedatives and/or neuromuscular
blocking agents. Cohen found the overall reliability
of the FOUR Score Scale to be excellent in this pop-
ulation. On the basis of these findings, we wanted to
expand on this area of work by modifying the original
FOUR Score Scale specifically for pediatric patients
of all age groups and across all developmental stages.
The purpose of this study was to determine whether
the Pediatric FOUR Score Scale (PFSS) could enhance
the clinical neurological assessment of pediatric in-
tensive care patients, including those intubated and/or
sedated, at our children’s hospital. This would be as-
sessed through analysis of the interrater reliability of
the PFSS used by nurses and the evaluation of the
validity of the PFSS in predicting morbidity, mortal-
ity, and long-term outcomes compared with the GCS.
Background
Loss of consciousness can occur any time there is an
interruption in blood flow or oxygenation to the brain,
because the brain is the most metabolically active
organ in the body, and it has no effective way to store
oxygen and glucose. Accurate assessments of the level
of consciousness lost are necessary in evaluating a
patient’s neurological status and therefore determin-
ing appropriate medical care (Cohen, 2009).
The GCS measures three aspects of consciousness:
eye, motor, and verbal responses. A numeric value is
given dependent on the responses, with the greatest
score being 15 and lowest score of 3 given to brain
death patients (Matis & Birbilis, 2008; Rabiu, 2011).
Although the GCS has been widely used in all as-
pects of healthcare, limitations exist, especially in the
ICUs because it was only tested and validated in adult
trauma, mostly head injury, patients. These limita-
tions include, but are not limited to, the following: (1)
the verbal component cannot be tested in intubated
patients; (2) sedated patients cannot respond appro-
priately; (3) it does not include brainstem reflexes or
changes in breathing patterns, which reflect severity
of coma; and (4) it does not include pediatric devel-
opmental milestones (Cohen, 2009; Matis & Birbilis,
2008; Rabiu, 2011; Wijdicks et al., 2005). Other scales
have been developed to overcome some of the lim-
itations of the GCS, but they have fallen short on
interrater reliability and are seldom used outside their
country of origin or area of medical specialty.
The FOUR Score Scale consists of four components
(eye response, motor response, brainstem reflexes, and
respiration). It also recognizes the locked-in syndrome
Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
These researchers ask whether the
Pediatric FOUR Score Scale (PFSS)
could enhance the clinical
neurological assessment of patients
cared for in pediatric intensive
care units.
and stages of herniation, providing families with a
better understanding of neurological predictive out-
come. The scale has been validated among the adult
population, and its interrater reliability and validity
have also been documented within multiple areas of
nursing, such as the neuro-ICU, medical ICU (MICU),
emergency department (ED), and pediatric populations
(Cohen, 2009; Iyer et al., 2009; Stead et al., 2009).
The original study (Wijdicks et al., 2005) included
120 adult intensive care patients, categorized to cover
all realms of the neurological population, and com-
pared the interrater reliability among neurointensive
nurses, neurology residents, and neurointensivists
to the GCS. The scale showed excellent interrater
reliability (.w = 0.82; 95% confidence interval [CI]
[0.77, 0.88]) and a high degree of internal consistency
(Cronbach’s ! = 0.86 for the first rater and 0.87 for
the second rater). To assess predictive validity of the
FOUR Score Scale, sensitivity and specificity of it
and the GCS were compared. The Modified Rankin
Scale was used as the ‘‘gold standard’’ to assess the
outcome of the patients, including in-hospital mortal-
ity and clinical diagnosis of brain death, and for com-
parison with the FOUR Score Scale and the GCS. The
FOUR Score Scale was also found to provide greater
neurological detail as compared with the GCS (Wijdicks
et al., 2005). These significant advantages included
elimination of the verbal component and incorpora-
tion of brain stem reflexes.
In 2008, Wijdicks published another cohort study
composed of 69 patients with neurological complaints
admitted in the ED. The raters using the FOUR Score
Scale and the GCS were ED physicians, residents,
and nurses. The overall kappa reported for the FOUR
Score Scale was 0.882, compared with 0.862 for the
GCS. Limitations included that approximately half of
the studied population consisted of ‘‘alert’’ patients;
therefore, incorporating more stuporous or comatose
patients would have been desirable (Stead et al., 2009).
Dr. Wijdicks expanded his studies in 2009 to other
populations of inpatients, including those in the MICU.
The raters for this study included nurses, fellows, and

ICU staff consultants. Excellent interrater reliability
among staff with a . value of 0.96 or greater was
reported. Also, the FOUR Score Scale correctly iden-
tified 100 MICU patients with neurological disability
of acute metabolic derangements, sepsis, and shock,
meaning it matched the outcome of their other as-
sessment criteria, the GCS. This study concluded that
the FOUR Score Scale was a good predictor of the
patient’s prognosis, even when expanding its use to
include critically ill patients, and again pointed out
many advantages over the GCS in another hospital
setting (Iyer et al., 2009).
Also in 2009, the FOUR Score Scale was validated
in a pediatric population. This application study fol-
lowed 60 neuroscience patients, aged 2Y18 years, over
a 1-year period. Patients who received sedatives and
neuromuscular blockades were excluded. Consistent
with the original 2005 study, the Modified Rankin
Scale was used to compare neurological functional
outcomes. The raters for this study were 35 pediatric
critical care nurses with clinical care experience ranging
from 1 to 40 years. Comparatively, the overall reliability
for the pediatric GCS was good (.w = 0.74, 95% CI
[0.59, 0.87]) but was found to be excellent for the
FOUR Score Scale (.w = 0.95, 95% CI [0.91, 0.99]).
Sensitivity and specificity were incorporated into the
calculation of the predictive validity (Cohen, 2009).
When doing a literature search to determine if the
FOUR Score Scale was right for our children’s hos-
pital, we noted that there was no current literature on
a PFSS and that there were some limitations to the
previous studies related to our specific patient popu-
lation. Our eight-bed pediatric ICU (PICU) has an
average census of 4.1 patients and a death rate of 2%
(per 100 patients), based on a total average of 503
patients over a 1-year period. The limitations we iden-
tified included our small neuroscience and brain death
population, our large sedated population, and various
developmental levels of patients.
Because of these various limitations, we planned to
modify the FOUR Score Scale for pediatric patients
with the goal of prospectively studying the PFSS
in ICU patients at our children’s hospital. Our current
practice for assessing a patient’s neurological status is
the use of the pediatric GCS for patients under the age
of 2 years and the original GCS for all other patients,
whereas the Richmond Agitation Sedation Scale (RASS)
score is used as a sedation scale. However, this prac-
tice does not reflect an accurate neurological assess-
ment, especially in intubated and/or sedated pediatric
individuals. The GCS was developed to assess the
level of consciousness of patients to aid in predicting
their neurological prognosis. However, it was only
tested and validated in adult patients with head in-
juries and requires a verbal response that cannot be
Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
Volume 46 & Number 2 & April 2014 81
obtained from intubated patients. The RASS relies on
a patient’s auditory and visual acuity, so it is not suit-
able for patients with severe impairments. It also does
not take into account sedation effects on neurological
status and was never validated in pediatric patients.
Thus, the PFSS would essentially combine these two
scales (GCS and RASS) into one, which could be used
to assess any pediatric patient despite their diagnosis
and developmental level and whether they are intu-
bated and/or sedated.
The PFSS includes (1) age appropriate responses
for children, inclusive of all developmental milestones;
(2) diagnosis, inclusive of sedated patients; and (3)
age-appropriate respiratory rates. We included five dif-
ferent developmental categories based on the American
Association of Critical Care Nurses guidelines: in-
fant (0Y12 months), toddler (1Y3 years), preschooler
(3Y5 years), school-age (6Y12 years), and adolescent
(13+ years; Slota, 2006a).
Healthy infants will track objects and open their
eyes spontaneously but will not follow direct com-
mands. They also have a higher respiratory rate and
an irregular breathing pattern compared with toddlers
and adults. Healthy infants can sit supported, crawl, or
walk depending on their age in months. They are also
nonverbal but communicate well in other ways such
as crying, smiling, and interacting with adults and other
children (Ely et al., 2003); however, they cannot make
the thumbs-up or peace-sign hand gestures. Toddlers
provide a little more information. They begin with
one-word sentences and expand their vocabulary quickly
in a short time. Healthy toddlers can follow simple
commands. Toddlers have a higher respiratory rate
than adults, but lower than infants, and tend to also
breathe in an irregular pattern; however, this does not
mean they have increased work of breathing (Ely
et al., 2003). Preschool and school-aged children can
take simple commands and incorporate more difficult
tasks. Adolescents are very similar to the adult pop-
ulation (Ely et al., 2003).
We also created the PFSS with the consideration
of children in each of these developmental categories
who are developmentally and/or physically disabled
(e.g., Down’s syndrome, cerebral palsy, paralysis).
Because the PFSS was modified from a developmen-
tal standpoint, our hopes were that the outcomes will
not vary from those seen with the original FOUR
Score Scale.
Normal respiratory rates for children were used
from the Pediatric Basic Life Support algorithm: infant,
G1 year, 30Y60 breaths per minute (bpm); toddler,
1Y3 years, 24Y40 bpm; preschool, 4Y5 years, 22Y34 bpm;
school-age, 6Y12 years, 18Y30 bpm; and adolescent,
13Y18 years, 12Y16 bpm (Berg et al., 2010). These
rates, along with the developmental categories, are
82 Journal of Neuroscience Nursing
part of a normal physical examination of a child’s
neurological system, which according to American
Association of Critical Care Nurses guidelines, also
includes general appearance, skull examination, level
of consciousness, motor function including reflexes,
sensory function, cerebellar function, cranial nerve
function, fundoscopic examination, and vital signs
(Slota, 2006b). The PFSS incorporates several of these
examination parameters.
In the original FOUR Score Scale study, the Mod-
ified Rankin Scale was used as the ‘‘gold standard’’ to
compare and contrast the adult GCS and the FOUR
Score Scale. The Modified Rankin Scale was devel-
oped and validated on adult patients and is primarily
used for stroke disability assessment (Wilson et al.,
2005). It is not commonly used on children, because
it lacks formal validation in pediatric populations.
Because we have modified everything for the pedi-
atric population, our ‘‘gold standard’’ was the Pediatric
Cerebral Performance Category (PCPC) Scale, which
was validated in the pediatric population. The PCPC
was developed and used as a tool to assess functional
morbidity and cognitive impairment after critical ill-
ness or injury in pediatric patients (Fiser, 1992). It
projects probable outcomes when more extensive psy-
chometric testing is not realistic or desirable and has
been found to be significantly related to the Pediatric
Risk of Mortality Score, morbidity, length of stay, and
total hospital charges (Fiser et al., 2000). Both the
Rankin and PCPC scales have also been tested and
validated in the past years inclusive of interrater reli-
ability (Fiser et al., 2000; Wijdicks et al., 2005). The
interrater reliability was showed as excellent in the
previous studies for the PCPC scale.
If the correlation is high between the two scales
(PFSS and PCPC) and low between the GCS and
PCPC, the PFSS will show association between pa-
rameters such as morbidity, length of stay, and sever-
ity of illness or injury and in turn disprove the GCS,
like previous studies cited (Fiser et al., 2000). There-
fore, the PFSS should potentially provide greater in-
formation than the formally used GCS when assessing
critically ill neurologically impaired pediatric patients.
Methods
Before creating the PFSS (Figure 1), consent from
Dr. Wijdicks to modify the original scale was obtained.
The modifications made to the adult FOUR Score
Scale to create the PFSS included (1) the removal of
‘‘or opened’’ from category 4 of the eye response.
This was done to allow better assessment of the very
young who cannot blink on command and certain de-
velopmentally disabled patients, such as those with
Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
cerebral palsy, whose eyes may be open but are not
neurologically intact. (2) The addition of ‘‘age appro-
priate spontaneous movement without stimulation’’
to category 4 of the motor response was done to allow
better assessment of all ages and developmental levels,
because very young patients and some others with de-
velopmental and/or physical disabilities cannot give
the thumbs-up or peace-sign hand gesture or make a
fist. (3) ‘‘Not intubated’’ and ‘‘age appropriate’’ were
added to the breathing pattern categories of the res-
pirations section. This was intended to clarify when no
intubation was in place and to point out that children
of different ages have differing respiration rates. These
modifications were made by the author (BLC) utilizing
the Guidelines for Age-Appropriate Assessment and
Nonpharmacologic Management of Pain (Slota, 2006b,
2006c) and then reviewed and approved by a pediatric
intensivist (CTS) and pediatric neurologist (MAK) based
on guidelines they follow from the American Academy
of Neurology. We realize that the changes made to the
adult FOUR Score Scale were minimal but felt they were
necessary to improve the assessments made in a pediatric
patient population, including those intubated and/or
sedated as well as developmentally and/or physically
disabled.
After institutional review board approval of this
prospective study, PICU nurses were asked for their
assistance in accomplishing the research. Eight PICU
nurses volunteered and were trained as ‘‘raters.’’ Clin-
ical PICU experience ranged from 1 year to 30 years.
The training included a 1-hour session for review of
the GCS scale (both the original for assessment of
children 2 to 19 years and the pediatric for assessment
of children up to 23 months of age), RASS, and PCPC
and overview of the PFSS. Each nurse was also given
an instructional card they could use as a reference
during their assessments. They were then given four
sample patients and asked to assess the patients using
the PFSS. After the training session, each nurse agreed
not to discuss their assessments with each other or any
other nurses, so as not to bias the nurses’ care of the
patients as well as affect the study outcomes. The
principal investigator expressed the importance of this
during the project, as we were studying the interrater
reliability of nurses.
Eighty patients were consented, consecutively en-
rolled, and participated in the study. These included a
variety of patients (cardiac, neurological, postoperative,
and trauma) who were admitted to the multidisciplinary
PICU, and patients could be intubated and/or receiving
sedative agents, such as lorazepam, fentanyl, morphine,
precedex, propofol, and versed. Patients receiving bar-
biturates, ketamine, and neuromuscular blockers were
excluded because of their effects on intracranial pres-
sure. Pregnant patients were also excluded.

FIGURE 1 Pediatric Four Score Scale
Consistent with the previous studies (Cohen, 2009;
Iyer et al., 2009; Stead et al., 2009), patients were
categorized into four neurological groups based on
their level of consciousness: (1) alert; (2) clouding of
consciousness: ‘‘reduced wakefulness, confusion, and
alternating drowsiness and hyperexcitablity’’; (3)
obtundation: ‘‘mild to moderate reduction in alertness,
reduced interest in the environment, and increased pe-
riods of sleep’’; and (4) stupor/coma: ‘‘unresponsive
except to vigorous and repeated stimuli or no verbal
or motor response to environmental stimuli’’ (Slota,
2006d). The nurses participating in the study assigned
each patient to a neurological category at the start of
each assessment session. This was done because a pa-
tient’s neurological status can change frequently, either
improving or declining, during their PICU stay and/or
because of any sedation that may be given. Therefore,
the same patient could have been assigned to more
than one category during their study participation.
Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
Volume 46 & Number 2 & April 2014 83
Seven trained nurses could have assessed each
subject during their PICU stay. Two different nurses
independently, yet simultaneously (or within minutes
of one another), assessed the PICU subjects using the
PFSS, GCS, and RASS on three different occasions
(between PICU admission and end of day 1, days 2Y3,
and day 4 or after, until PICU discharge). The nurses
were not allowed to assess the same patient twice, and
the nurseYrater teams and order in which the assess-
ments (PFSS, GCS, RASS) were performed varied to
reduce bias. The data were then compared with the
PCPC as ‘‘the point of reference.’’ The PCPC of each
subject was assessed by only one nurse (who did not
use the PFSS, GCS, or RASS) at PICU admission and
discharge to ensure consistency and to prevent the other
nurses doing the PFSS, GCS, and RASS assessments
from being biased by the PCPC data, which could
also influence patient care. Outcome was assessed
using the PCPC scale: 1 = normal, 2 = mild disability,
84 Journal of Neuroscience Nursing
3 = moderate disability, 4 = severe disability, 5 = coma
or vegetative state, and 6 = brain death (Fiser et al.,
2000).
Statistical Analysis
Descriptive statistics were used to summarize the pa-
tients’ demographic and clinical characteristics. To
estimate interrater agreement, each patient (at several
different time points) received a pairwise independent
test by two different raters, and the agreement of the
first and second test (of each score and total score for
the PFSS, GCS, and RASS) was used. Weighted kappa
(.w) values and their 95% CI as well as standard error
(SE) were calculated to evaluate the reliabilities for
the PFSS, GCS, and RASS. In our study, a .w statistic
of greater than 0.80 was considered excellent agree-
ment. Cronbach’s alpha coefficients were computed
to assess the internal consistencies of the PFSS and
GCS. Spearman’s correlation coefficient between the
PFSS and the GCS was calculated to assess the cri-
terion validity of the PFSS.
To assess the predictive validity of the PFSS, sen-
sitivity and specificity of the PFSS and GCS were
compared with the PCPC scale for the prediction of
poor outcomes and in-hospital mortality at PICU ad-
mission and discharge. The area under the receiver
operating characteristic (ROC) curve (AUC) and its
95% CI were calculated for PFSS and GCS using the
average rating of the two nurses at each assessment
time point. An ROC analysis curve compares the sen-
sitivity (true rate of the outcome) with the false rate
(1 = specificity), and as an AUC gets closer to 1.00,
it indicates that the assessment tool is better able to
identify the outcome state (Schonjans, 2008).
Results
Of the 80 patients who participated in the study, 78
had a total of 121 paired-wised ratings and were used
in the analysis. The average age of the patients was
6.34 years, and there were seven adverse events (deaths).
Nine of the seventy-eight patients (11.5%) had a de-
velopmental and/or physical disability (i.e., autism,
cerebral palsy, cerebral vascular accident, GuillameY
Barre syndrome, spinal muscular atrophy type II).
There were a total of 115 records categorizing the pa-
tients into the four neurological groups, with the dis-
tribution as follows: Alert, 62 patient times (53.91%);
clouding of consciousness, 13 patient times (11.30%);
obtundation, 27 patient times (23.48%); and stupor/
coma, 13 patient times (11.30%). Other demographics
and clinic outcomes are shown in Table 1.
The overall reliability was excellent for both the
PFSS (.w = 0.89, 95% CI [0.83, 0.94]) and the GCS
(.w = 0.89, 95% CI [0.84, 0.94]), whereas the
.w of RASS was 0.67, which suggests good interrater
Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
TABLE 1. Demographics and Some
Clinical Outcomes for
78 Patients
Age (mean T SD), years 6.34 T 6.22; median = 3.3
(range: 0.1Y18.9)
Infant (0Y12 months) 32 (41.03%)
Toddler (1Y3 years) 13 (16.67%)
Preschool (3Y5 years) 5 (6.41%)
School (6Y12 years) 14 (17.95%)
Adolescent (13+ years) 14 (17.95%)
Male 45 (57.69%)
Current intubation 34 (43.59%)
Death 7 (8.97%)
Adverse event 7 (8.97%)
Sedation 31 (39.74%)
agreement (Table 2). Table 2 also shows the reliabil-
ities classified by intubation and day of PICU hospi-
talization. Cronbach’s alpha coefficient showed good
internal consistency for the PFSS (Cronbach’s alpha =
0.78 for the first rating and 0.79 for the second rating)
and the GCS (Cronbach’s alpha = 0.76 for the first
rating and 0.77 for the second rating). Spearman’s
correlation coefficients between the GCS and PFSS
were high ( p = .87 for the first rating and .89 for the
second rating).
ROC analysis curves were estimated to compare
prediction of poor outcome (defined as PCPC scores
of 4Y6) between the PFSS and the GCS. The AUC for
the PFSS and GCS total scores were 0.9043 and 0.9054,
respectively. The AUC of the PFSS was almost the
same as for the GCS by comparing 0.9043 with 0.9054
(p = 0.9552). Sensitivity and specificity were 0.9333
and 0.7903, respectively, for a PFSS total score of 12
and 0.9333 and 0.8387, respectively, for a GCS total
score of 9 (Figure 2). Unfortunately, a separate anal-
ysis could not be done to predict outcome in only the
sedated and/or intubated patients because of the small
sample sizes, n = 31 and n = 34, respectively.
To compare prediction of in-hospital mortality be-
tween the PFSS and the GCS, additional ROC analysis
curves were estimated. The AUC for the PFSS and
GCS total scores were 0.9296 and 0.9095, respectively.
The AUC of the PFSS was slightly better than for the
GCS by comparing 0.9296 with 0.9095 (p = 0.2113),
which did not reach the statistical significance level
of .05, and may be because of the small sample size
(only seven death events) in this study. Sensitivity and
specificity were 0.8574 and 0.9437, respectively, for
a PFSS total score of 7 and 0.8571 and 0.9296, re-
spectively, for a GCS total score of 6 (Figure 3).
 TABLE 2. Weighted Kappa Values, Standard Error, and 95% Confidence Intervals for Interrater Agreement on the Pediatric
FOUR Score Scale (PFSS), Glasgow Coma Scale (GCS), and Richmond Agitation Sedation Scale (RASS)
PFSS GCS
Eyes Motor Brain Stem Reflexes Respiration Total Eyes Motor Verbal Total RASS
For the overall, N = 121 pairwise ratings
Weight . 0.94 0.86 0.74 0.88 0.89 0.90 0.90 0.88 0.89 0.67
SE 0.02 0.05 0.12 0.03 0.03 0.03 0.03 0.04 0.03 0.07
95% CI 0.90, 0.98 0.76, 0.96 0.50, 0.98 0.81, 0.94 0.83, 0.94 0.84, 0.96 0.84, 0.96 0.81, 0.95 0.84, 0.94 0.53, 0.80
For the not intubated, N = 64 pairwise ratings
Weight . 0.88 0.96 1.00 0.72 0.85 0.90 0.88 0.87 0.89 0.80
SE 0.08 0.04 0.00 0.10 0.05 0.06 0.08 0.07 0.05 0.08
95% CI 0.72, 1.00 0.88, 1.00 1.00, 1.00 0.53, 0.91 0.75, 0.95 0.79, 1.00 0.73, 1.00 0.74, 1.00 0.79, 0.99 0.64, 0.96
For the intubated, N = 57 pairwise ratings
Weight 0 0.94 0.80 0.76 0.64 0.82 0.86 0.89 0.56 0.80 0.51
SE 0.02 0.07 0.12 0.11 0.06 0.05 0.04 0.15 0.05 0.10
95% CI 0.90, 0.99 0.66, 0.94 0.52, 1.00 0.42, 0.85 0.71, 0.93 0.76, 0.96 0.82, 0.96 0.27, 0.84 0.70, 0.91 0.30, 0.71
For days 0Y1, N = 51 pairwise ratings
Weight . 0.95 0.85 0.90 0.85 0.90 0.96 0.92 0.91 0.93 0.73
SE 0.04 0.09 0.09 0.06 0.04 0.03 0.04 0.04 0.04 0.09
95% CI 0.88, 1.00 0.67, 1.00 0.74, 1.00 0.73, 0.96 0.82, 0.97 0.91, 1.00 0.83, 1.00 0.82, 1.00 0.85, 1.00 0.55, 0.90
For days 2Y3, N = 41 pairwise ratings
Weight . 0.91 0.82 0.58 0.85 0.87 0.83 0.90 0.81 0.85 0.56
SE 0.04 0.09 0.22 0.07 0.03 0.07 0.04 0.08 0.05 0.12
Volume 46

95% CI 0.84, 0.98 0.65, 1.00 0.15, 1.00 0.72, 0.98 0.82, 0.93 0.69, 0.96 0.81, 0.99 0.65, 0.96 0.75, 0.94 0.31, 0.80
&

For day Q 4, N = 29 pairwise ratings

Weight . 0.97 0.93 N/Aa 0.96 0.84 0.93 0.87 0.90 0.89 0.75
SE 0.03 0.07 N/A 0.03 0.10 0.05 0.08 0.07 0.06 0.14
Number 2

95% CI 0.92, 1.00 0.80, 1.00 N/A 0.90, 1.00 0.64, 1.00 0.83, 1.00 0.71, 1.00 0.76, 1.00 0.78, 1.00 0.48, 1.00
&

Note. CI = confidence interval; N/A = not applicable.

a
All 29 pairwise ratings had the exact same score of 4.
April 2014

Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
85
86 Journal of Neuroscience Nursing

Discussion FIGURE 3 Receiver Operating

The results of this study show that the PFSS is ex- Characteristic Analysis of PFSS
cellent for interrater reliabilities and for prediction of Total Score and GCS Total
poor outcome and in-hospital mortality in a pediatric Score for Predicting
population. The .w values in the excellent range in- In-Hospital Mortality
dicate that there is homogeneity in the nurse raters’
use of the PFSS, suggesting that it is an easy assess-
ment tool that can be used reliably and consistently by
nurses of varying experience levels. It can be used in
sedated and/or intubated patients and at varying times
during a patient’s hospitalization based on the weighted
kappa values presented in Table 2. The PFSS’s highly
accurate prediction value for poor outcome and in-
hospital mortality makes it an informative tool for
neurological assessments.
However, the study failed to show that the PFSS is
better than the GCS. The results did not indicate any
statistically significant difference in the interrater re-
liabilities for nurseYrater pairs under various situations
between the PFSS and the GCS, and the prediction
analyses were quite similar when correlated to the
PCPC. This was similar to Cohen’s (2009) study that
showed that the original FOUR Score Scale and GCS
were comparable in predicting outcome in their pedi-
atric study population.
Although our study did not show that the PFSS is
better than the GCS, it also did not indicate that it
was any worse than the GCS, which has only been validated in adult head trauma patients. Other benefi-
cial findings from our study are that our patient
population included (1) intubated and/or sedated
patients and that the weighted kappa values for
FIGURE 2 Receiver Operating Characteristic intubated and nonintubated patients were similar and
Analysis of PFSS Total Score and (2) patients of all developmental stages, including
GCS Total Score for Predicting those with developmental disabilities (e.g., Down’s
Poor Outcome (Defined as syndrome) and physical disabilities (e.g., Cerebral
PCPC Scores of 4Y6) palsy, paralysis). Hence, it can be used across a broad
pediatric patient population. This study also expands
on the work done by Cohen (2009).
The largest limitation of our study was the small
sample size. Additional limitations included changes
in the staffing of nurses and a large turnover rate of
patients; some patients were only assessed one time
and then discharged to the regular pediatric floor or
home. Also, ~54% of our population was assessed to
the ‘‘alert’’ neurological category. In addition, the use
of sedative medications, including benzodiazepines
and opiates, alters the brain causing central nervous
system depression and can affect each patient differ-
ently. However, there was still correlation between the
PFSS and GCS in sedated patients.

Conclusion/Summary
Enhancing the neurological assessment of a pediatric
patient by the nursing community allows for the

Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.

early intervention of medical emergencies, whether this
means an immediate computed tomography or magnetic
resonance imaging, additional laboratory values, the
placement of an intracranial monitoring device, or
emergency surgery. Also, having the potential to provide
greater knowledge of the neurological assessment in se-
dated and/or intubated patients, thereby decreasing the
number of ventilator days, incidence of ventilator-
associated pneumonia, and accidental extubations,
would be of benefit. This would also allow the nursing
staff to provide the parents of patients who are
intubated and/or sedated with a better understanding
of their child’s neurological status. Therefore, having
an assessment tool that is easy to use, has excellent
interrater reliability, and can accurately predict out-
comes in a broader pediatric patient population could
greatly improve on the quality of patient care.
The demonstration of the reliabilities of the PFSS
used by a nursing staff was a step in the right direction
in getting this assessment tool noticed by the medical
community that cares for pediatric neurological pa-
tients. The next step is to evaluate the PFSS in a larger
number of patients across a broader spectrum of the
pediatric population (i.e., range of ages, intubated and/or
sedated, various developmental and/or physical dis-
abilities), so it can eventually be accepted for use.
Acknowledgments
The authors dedicate this article to Dr. C. Todd Stewart,
who tragically passed away after its acceptance. It was
an honor and a pleasure to work with him for so many
years on this project. Without his dedication, this
article would not have been possible.
The authors thank Monica A. Koehn, MD,
Department of Neurology (Pediatric Division) for
her expert review of the PFSS and the Marshfield
Clinic Research Foundation’s Office of Scientific
Writing and Publication for assistance in the prepara-
tion of this manuscript.
References
Berg, M. D., Schexnayder, S. M., Chameides, L., Terry, M.,
Donoghue, A., Hickey, R.W., I Hazinski, M. F. (2010).
Part 13: pediatric basic life support: 2010 American Heart
Association Guidelines for Cardiopulmonary Resuscitation
and Emergency Cardiovascular Care. Circulation, 122(18
Suppl 3), S862YS875.
Cohen, J. (2009). Interrater reliability and predictive validity of
Copyright © 2014 American Association of Neuroscience Nurses. Unauthorized reproduction of this article is prohibited.
Volume 46 & Number 2 & April 2014 87
the FOUR Score Coma Scale in a pediatric population. Journal
of Neuroscience Nursing, 41(5), 261Y267.
Ely, E. W., Truman, B., Shintani, A., Thomason, J. W.,
Wheeler, A. P., Gordon, S., I Bernard, G. R. (2003).
Monitoring sedation status over time in ICU patients: The
reliability and validity of the Richmond Agitation Sedation
Scale (RASS). Journal of the American Medical Association,
289(22), 2983Y2991.
Fiser, D. H. (1992). Assessing the outcome of pediatric intensive
care. Journal of Pediatrics, 121(1), 68Y74.
Fiser, D. H., Long, N., Roberson, P. K., Hefley, G., Zolten, K.,
& Brodie-Fowler, M. (2000). Relationship of pediatric over-
all performance category and pediatric cerebral performance
category scores at pediatric intensive care unit discharge with
outcome measures collected at the hospital discharge and
1- and 6-month follow-up assessments. Critical Care Medicine,
28(7), 2616Y2620.
Iyer, V. N., Mandredar, J. N., Danielson, R. D., Zubkov, A. Y.,
Elmer, J. L., & Wijdicks, F. M. (2009). Validity of the
FOUR score coma scale in the medical intensive care unit.
Mayo Clinic Proceedings, 84(8), 694Y701.
Matis, G., & Birbilis, T. (2008). The Glasgow Coma ScaleVA
brief review. Past, present, future. Acta Neuroligca Belgica,
108(3), 75Y89.
Ommaya, A. K. (1966). Trauma to the nervous system. Annals of
the Royal College of Surgeons of England, 39(6), 317Y347.
Rabiu, T. B. (2011). Revisiting the eye opening response of the
Glasgow Coma Scale. Indian Journal of Critical Care Medicine,
15(1), 58Y59.
Schonjans, F. Receiver operating characteristic curve analysis
[software]. Mariakerke, Belgium: MedCalc.
Slota, M. C. (2006a). Childhood development: Psychosocial, emo-
tional, cognitive. Core curriculum for pediatric critical care nurs-
ing (2nd ed., pp. 4Y6). St. Louis, MO: SaundersYElsevier.
Slota, M. C. (2006b). Physical examination. Core curriculum for
pediatric critical care nursing (2nd ed., pp. 337Y348). St. Louis,
MO: SaundersYElsevier.
Slota, M. C. (2006c). Guidelines for age-appropriate assessment
and nonpharmacologic management of pain. Core curricu-
lum for pediatric critical care nursing (2nd ed., pp. 14Y17).
St. Louis, MO: SaundersYElsevier.
Slota, M. C. (2006d). Level of consciousness. Core curriculum
for pediatric critical care nursing (2nd ed., pp. 338Y339).
St. Louis, MO: SaundersYElsevier,
Stead, L. G., Wijdicks, F. M., Bhagra, A., Kashyap, R.,
Bellolio, M. F., Nash, D. L., I William, B. (2009). Validation
of a new coma scale, the FOUR Score, in the emergency
department. Neurocritical Care, 10(1), 50Y54.
Teasdale, G., & Jennett, B. (1974). Assessment of coma and
impaired consciousness. A practical scale. Lancet, 2(7872),
81Y84.
Wijdicks, E. F., Bamlet, W. R., Maramattom, B. V., Manno, E. M.,
& McClelland, R. L. (2005). Validation of a new coma scale:
The FOUR Score. Annals of Neurology, 58(4), 585Y593.
Wilson, J., Hareendran, A., Grant, M., Hendry, A., Potter, J.,
Bone, I., & Muir, K. W. (2005). Reliability of the modified
Rankin scale across multiple raters: Benefits of a structured
interview. Stroke, 36(4), 777.