Reliability and validity of the Harris Infant

Neuromotor Test
Susan R. Harris, PhD, PT, and Linda E. Daniels, MA, OTR, OTC
are suspected of having a developmen-
Objective: To examine the reliability and validity of the Harris Infant Neu- tal concern from those who appear to
romotor Test (HINT), a screening tool that can be administered and scored be developing normally.2 Those with
in <30 minutes, with the goal of identifying neuromotor differences in in- suspected neuromotor or cognitive
fants aged 3 to 12 months. concerns are then referred for more
comprehensive developmental assess-
Study design: Infants, aged 3 to 12 months (n = 54), were assessed in 2
ments, such as the Bayley Scales of In-
high-risk infant follow-up programs in Vancouver, British Columbia. Inter-
fant Development-II.3
rater, test-retest, and intra-rater reliability were examined. Concurrent and
predictive validity of the HINT with the Bayley Scales of Infant Develop- BSID Bayley Scales of Infant Development
ment-II (BSID-II) were evaluated by using the Pearson product-moment BSID-II Bayley Scales of Infant Development-II
HINT Harris Infant Neuromotor Test
correlation. ICC Intraclass correlation coefficient
Results: Intraclass correlation coefficients for reliability for the Total
HINT Score ranged from 0.98 to 0.99. Concurrent validity of the HINT Criteria for evaluating screening
with the BSID-II Mental Scale during the first year was r = –0.73 (P < .01), tests include acceptability (to the chil-
and with the BSID-II Motor Scale, r = –0.89 (P < .01). The predictive rela- dren and family being screened, the
tionships between the HINT and the BSID-II at 17 to 22 months were professionals receiving the referrals,
r = –0.11 for the BSID-II Mental Scale and r = –0.49 (P < .01) for the and the community at large), simplici-
ty (in learning the test and in its ad-
BSID-II Motor Scale.
ministration), cost, appropriateness (to
Conclusions: The HINT is reliable for screening infant neuromotor perfor- the population under consideration),
mance and has strong concurrent validity with the Bayley-II Mental and reliability, and validity.4
Motor Scales. HINT scores during the first year accounted for 24% of the The Harris Infant Neuromotor Test
variance of Bayley-II Motor scores at 17 to 22 months. (J Pediatr 2001;139: is being developed as a simple, nonin-
249-53) vasive, reliable, and valid screening
tool for use in either clinical or re-
search settings to identify neuromotor
Although approximately one half of ture birth (ie, two thirds of infants with differences in infants aged 3 to 12
children with cerebral palsy were born later cerebral palsy) or prenatal sub- months. Designed to be administered
prematurely1 and thus are more apt to stance exposure, are seldom screened in <30 minutes, the HINT is intended
be assessed in high-risk infant follow- or tested for developmental concerns. for use by occupational therapists,
up clinics, those infants who are not Most pediatric screening tests are physical therapists, pediatricians, fam-
deemed “at risk,” either from prema- designed to discriminate children who ily physicians, and community health
nurses. Item development of the HINT
From the School of Rehabilitation Services, Faculty of Medicine, University of British Columbia, Vancouver, British
Columbia, Canada.
was based on extensive earlier re-
Supported in part by a grant from the British Columbia Health Research Foundation (No. 146- search on early identification of neu-
95-1). rodevelopmental problems.5-15
Submitted for publication Aug 23, 2000; revisions received Dec 13, 2000, and Feb 23, 2001; The first development edition of the
accepted Mar 15, 2001.
HINT16 was used in a pilot study to
Reprint requests: Susan R. Harris, PhD, PT, School of Rehabilitation Sciences, Faculty of Med-
assess 31 high-risk infants, aged 3 to 9
icine, T325-2211 Wesbrook Mall, University of British Columbia, Vancouver, British Columbia
V6T 2B5, Canada. months.17 A series of 4 questions was
Copyright © 2001 by Mosby, Inc. posed to the primary caregivers about
0022-3476/2001/$35.00 + 0 9/21/115896 how their infants were faring develop-
doi:10.1067/mpd.2001.115896 mentally. Their responses were used to

249
 HARRIS AND DANIELS
compare their perceptions of the in-
fant’s development with test results on
the Bayley Scales of Infant Develop-
ment.18 A high degree of concurrence
was found, particularly between the
caregivers’ judgments in response to the
HINT questions and the therapist’s test
results with the Bayley Motor Scale
(sensitivity = 80%; specificity = 90.9%).
To examine the content validity of
the HINT, that is, the “representative-
ness” of the HINT items to the concept
of neuromotor development, the first
development edition was distributed to
an international panel of 26 experts for
assessment, review, and modifications.
The revised edition of the HINT
(1993) was used in the current study.19
The 5-item parent/caregiver section
is designed to assess the parents’ level
of concern and the specific areas of
concern that they may have about their
child. The infant assessment section
(items 1-21) includes motor behaviors
in a supine-lying position, during tran-
sition to a prone-lying position, in a
prone-lying position, and in supported
sitting and supported standing posi-
tions. The items are ordered in a logical
sequence, by position, to minimize in-
fant handling. Head circumference is
measured also, as well as behavioral
state and stereotypical behaviors. A 2-
part developmental and qualitative
judgement item (item 22) concludes the
test and is designed to summarize the
examiner’s overall clinical impressions
about the developmental appropriate-
ness of the infant’s neuromotor perfor-
mance and the quality of the infant’s
movement behaviors.19 The purpose of
the present study was to examine the
reliability and the criterion-related va-
lidity of the HINT in assessing infants
at risk for neuromotor and/or cognitive
difficulties.
METHODS
Subjects and Sites
A convenience sample of 54 high-
risk infants was recruited, primarily
from 2 developmental follow-up pro-
250
grams in Vancouver, British Columbia,
Canada. The Infants and Children at
Risk Program at Sunny Hill Health
Centre for Children provided longitu-
dinal developmental assessments for
infants and young children who were
at risk because of prenatal exposure to
drugs and/or alcohol. The Neonatal
Follow-Up Programme at British Co-
lumbia Children’s Hospital provides
systematic, comprehensive develop-
mental assessments for extremely low
birth weight infants (<1000 g), as well
as term infants with other biologic risk
factors such as neonatal seizures, hy-
drocephalus, or intrauterine growth
retardation. Sixty-one percent of the
infants in the sample were from Sunny
Hill and 37% were from British Co-
lumbia Children’s Hospital. One addi-
tional premature infant was recruited
from the Outreach Therapy Pro-
gramme in Port Alberni, British Co-
lumbia. Children with known congeni-
tal anomalies were excluded. The
study was approved by the Clinical
Research Ethics Board at the Univer-
sity of British Columbia, as well as by
the ethics committees of both hospitals.
Informed consent was obtained from a
parent or guardian of each infant.
Nineteen female and 35 male infants
ranged in chronological age from 3.07
months to 12.30 months (mean = 7.79
months) at the initial assessment, with
age corrected for prematurity in in-
fants born at <38 weeks’ gestation. Al-
most two thirds (63.5%) were born be-
fore term (23 to 37 weeks’ gestation),
and 51.9% had low birth weight
(<2500 g). Mean birth weight was
2494 g; mean gestational age was
35.67 weeks. Race of the infant’s
mother was white (43.1%), Native In-
dian/First Nations (25.5%), Asian
(13.7%), Hispanic (2%), East Indian
(2%), or other/unknown (13.7%).
Outcome assessments took place be-
tween 17.53 and 22.23 months of
chronological age (mean age = 18.95
months). Some subjects were not able
to keep all assessment appointments,
so the number of subjects and group
THE JOURNAL OF PEDIATRICS
AUGUST 2001
composition were not identical for
each aspect of the study.
Examiners and Examiner
Training
The assessments were conducted by
3 physical therapists and 2 occupation-
al therapists. In addition to pediatric
experience, the therapists had training
and experience in administering and
scoring the Movement Assessment of
Infants5 and the first edition of the
BSID.18 The senior author trained the
other testers by assessing 3 typically
developing infants (aged 5.5 to 7
months) while the other 4 examiners
observed and scored the test. In addi-
tion, training and inter-rater reliability
evaluation were conducted on the
BSID-II3 by the senior author. Six in-
fants, ranging in age from 31⁄2 to 18
months and without any risk factors,
were assessed on the BSID-II with
inter-observer reliability conducted by
each of the other assessors. Each of the
5 assessors served as the primary ex-
aminer for test administration and
scoring for at least one infant, with the
other 4 assessors observing and scor-
ing the test independently. Inter-rater
reliability for raw scores on the BSID-
II, across the 5 assessors, was obtained
at acceptable levels for the study to
proceed: intraclass correlation coeffi-
cient (2,1) = .993 for the Mental Scale;
ICC (2,1) = .995 for the Motor Scale.
Instruments
The HINT was administered by 1 of
the 5 assessors during the first year of
the infant’s life (3 to 12 months). The
first part of the HINT includes back-
ground information on the child and the
caregiver. The second part is a 5-item
parent/caregiver questionnaire that is
designed to assess specific areas of con-
cern. The third part, the Infant Assess-
ment, includes the 21 items that are ob-
served and scored by the test examiner,
such as motor behaviors in a supine-
lying position; motor behaviors during
transition from a supine-lying to a
prone-lying position; motor behaviors in
THE JOURNAL OF PEDIATRICS
VOLUME 139, NUMBER 2
a prone-lying position; motor behaviors
in a sitting position; observation of loco-
motion, posture, movements, and be-
havioral state; and measurement of head
circumference. After calculation of the
infant’s total HINT score based on sum-
ming the results of the 21 assessment
items, the examiner makes two summa-
ry decisions about the infant’s overall
neuromotor performance, one for the in-
fant’s developmental level and one for
the quality of motor performance.
In the current study, the Mental and
Motor Scales of the BSID-II were ad-
ministered. Raw scores are converted
to index scores. The Mental Develop-
mental Index and Psychomotor Devel-
opmental Index both have means of
100 and SDs of 15. Although an in-
creasing raw score on the HINT indi-
cates increasing risk, an increasing raw
score on the BSID-II represents im-
proved or more mature performance.
Procedures
To avoid bias, the HINT was adminis-
tered first. Background information was
first recorded from the chart. The
BSID-II Mental Scale was adminis-
tered next, followed by the Motor Scale.
To examine inter-rater reliability, 28
infants were assessed by 1 of the 5
therapists, designated as the primary
examiner, while a second therapist ob-
served and scored the test. The second
examiner independently administered
the 2 items that involve assessment of
muscle tone (passive range of motion
in supine position and passive range of
motion in prone position).
To evaluate test-retest reliability, the
primary examiner re-administered the
HINT to 30 infants within a period of
2 to 9 days without referring to the
original test sheet and scores.
Intra-rater reliability was assessed
by videotaping the administration of
the initial HINT assessment for 20 in-
fants in the sample. The videotapes
were later viewed twice by each of the
5 therapists at an interval of ≥1 month.
The assessors were blinded to their
previous scores.
Concurrent validity was examined by
comparing the infants’ HINT scores
with the raw scores on the BSID-II
Mental and Motor Scales, which were
administered during the same day in
the infant’s first year of life. Predictive
validity was evaluated by comparing
the HINT scores during the infant’s
first year with the raw scores on the
BSID-II Mental and Motor Scales
during the second year of life (17 to 22
months). BSID-II raw scores were
used because they represent the most
similar psychometric counterpart to the
HINT total raw scores. To avoid possi-
ble expectation bias, examiners were
blinded to all previous test scores dur-
ing administration of the later BSID-II.
Data Analysis
The ICC (2,1) was used to analyze
inter-rater, test-retest, and intra-rater
reliability for the total HINT score.20-22
An ICC of ≥0.80 was set for inter-rater,
test-retest, and intra-rater reliability as
“acceptable,” although ≥0.90 is consid-
ered desirable.23
To examine concurrent and predic-
tive validity of the total HINT score,
the Pearson r was used to express the
degree of relationship between the
HINT and BSID-II (Mental and
Motor Scales). A level of r ≥ 0.60 was
set for concurrent validity of the
HINT with the Mental and Motor
Scales of the BSID-II. This level was
set based on previous research exam-
ining concurrent validity between the
BSID Mental Scale and the Stanford-
Binet IQ test18 for a sample of 2-year-
olds. A correlation of r = 0.57 was re-
ported as a “substantial degree of
agreement” between the 2 tests. To ex-
amine the predictive validity of the
HINT to the later BSID-II, an r value
of ≥0.30 was hypothesized between the
first-year HINT and the second-year
BSID-II Mental and Motor Scales.
This level was set based on results of
previous research examining the pre-
dictive validity of the Movement As-
sessment of Infants for BSID scores at
1 and 2 years of age.7
HARRIS AND DANIELS
RESULTS
The ICC for inter-rater reliability for
the 28 infants was 0.99 for the Total
HINT score. Of the 21 individual
HINT items, the ICC was ≥.80 for 9
items (42.8%) and ≥.70 for 18 items
(85.7%). However, for 2 HINT items
(eye muscle control and stereotypical
behaviors), the ICCs were <.15 (com-
plete data available from authors on re-
quest).
For the 20 infants who were each test-
ed twice within 9 days, the ICC value
for test-retest reliability of the total
HINT score was 0.98. For the 21 indi-
vidual HINT items, the ICC was ≥.80
for 12 items (57.1%) and ≥.70 for 14
items (66.7%). For 2 HINT items, how-
ever, the test-retest reliability was very
poor (eye muscle control = 0.000; stereo-
typical behaviors = 0.377) (complete
data available from authors on request).
Intra-rater reliability was assessed
for each of the 5 therapists who twice
viewed videotapes of 20 infants, ap-
proximately 1 month apart in time.
Across the 5 therapists, ICC values
ranged from 0.98 to 0.99 for the Total
HINT score (complete data available
on request). Our hypotheses were thus
supported; that is, all ICC reliability
values were ≥.80.
The relationships of the Total HINT
score to raw scores on the BSID-II
Mental Scale and BSID-II Motor
Scale were calculated by using the
Pearson product-moment correlation.
These 3 scores were available for all 54
infants in the sample. Because the
HINT is predominantly a neuromotor
screening tool, it is not surprising that
it correlated more strongly with the
BSID-II Motor Scale (r = –0.89, P <
.01) than with the BSID-II Mental
Scale (r = –0.73, P < .01).
To examine the predictive validity of
the HINT during year 1 (3 to 12
months of age), the relationships of the
total HINT scores to the raw scores on
the BSID-II Mental and Motor Scales,
as administered during year 2 (17 to 22
months of age), were calculated. The
251
 HARRIS AND DANIELS
predictive relationship of the HINT to
the BSID-II Mental Scale was r =
–0.11. Our hypothesis was not sup-
ported because the correlation coeffi-
cient was <0.30. In contrast, however,
the predictive relationship between the
HINT and the BSID-II Motor Scale
was r = –0.49 (P < .01).
DISCUSSION
The inter-rater, test-retest, and intra-
rater reliabilities of the total HINT
score were all ≥0.98, well above the ac-
ceptable “benchmark” of 0.80 and also
exceeding the desirable level of
0.90.3,5,23 Of concern, however, were
the low inter-rater and test-retest relia-
bilities of 2 items: eye muscle control
and stereotypical behaviors. These low
coefficients were due, in part, to the
fact that these behaviors were seldom
observed among the infants in our
sample. Consequently, these items will
most likely be omitted from the final
version of the tool.
The moderately strong and signifi-
cant relationship between scores on
the HINT and the BSID-II Mental
Scale suggests that the HINT may be
tapping early cognitive behaviors, at
least as they are assessed during the
first year of life. The concurrent validi-
ty between the HINT and the BSID-
II Motor Scale (r = 0.89) is somewhat
lower than the correlation coefficient
reported by Piper and Darrah24 be-
tween the Alberta Infant Motor Scale
and the first edition of the BSID17 for
their sample of 48 at-risk infants (r =
0.98). The concurrent validity of the
Alberta Infant Motor Scale with the
BSID Mental Scale was not evaluated,
so it is impossible to make comparisons
with results from the present study.
The concurrent validity of the
BSID-II Mental Scale with the HINT
compares favorably with relationships
between the BSID-II and other
well-known, standardized assessment
tools.25-27 Furthermore, the HINT
concurrent validity values compared
252
with each of the BSID-II scales exceed
the minimally acceptable level of 0.60,
according to Meyer.28
In light of the fact that the HINT is
quick, inexpensive, and easy to adminis-
ter, it is encouraging that this screening
test provides reasonably comparable in-
formation to the more time-intensive and
costly scale. However, one important dif-
ference in the studies reported in the
BSID-II manual, as compared with the
current study, is that the former studies
used index scores, whereas the present
study used raw scores. Raw scores were
preferable for this study because they
represented a form of measurement par-
allel to the HINT raw scores.
The predictive validity of the early
HINT (3-12 months) compared with
the later BSID-II Motor Scale (17-22
months), although relatively modest (r
= –0.49, P < .01), is stronger than the
relationship between the BSID-II
Motor Scale administered during the
first year of life and that same scale ad-
ministered during the infant’s second
year of life (r = 0.34, P < .05). Although
the early HINT accounted for nearly
one quarter (24%) of the BSID-II
Motor outcome during year 2, the
early BSID-II Motor Scale accounted
only for 11.6% of the later BSID-II
Motor outcome.
The fact that the HINT was superior
to the BSID-II Motor Scale at predict-
ing later motor development on the
BSID-II Motor Scale speaks to its fu-
ture utility as a screening test, especial-
ly for infants at risk for neuromotor
difficulties.
After collection of normative data on
300 healthy Canadian infants (2000-
2002), including BSID-II data on 60
infants during the first year and at ap-
proximately 18 months, classification
analyses will be conducted to examine
the sensitivity, specificity, and positive
and negative predictive values of the
HINT compared with the later BSID-II
scores. Data from the 60 healthy infants
will be combined with those from the 54
high-risk infants in the present study.
This will provide another, widely ac-
THE JOURNAL OF PEDIATRICS
AUGUST 2001
cepted method for examining the
HINT’s predictive validity.29 Because
we currently have data only on high-risk
infants, it would be premature to con-
duct classification analyses at this point.
The appropriateness of the HINT to
the population under consideration was
also evaluated in the content validity
study.19 Although 100% of the experts
rated the HINT as average or above av-
erage in its ability to screen for early
neuromotor delays, only 26.3% gave
similar ratings when asked about the
HINT’s ability to screen for early cogni-
tive delays. Results of the present study,
particularly with regard to the HINT’s
ability to predict later cognitive perfor-
mance, corroborate these opinions.
The final two criteria for evaluating
screening tests are reliability and valid-
ity.4 The HINT has strong inter-rater,
test-retest, and intra-rater reliability, as
well as strong concurrent validity with
the BSID-II. The HINT’s predictive
validity for later motor performance,
although modest, appears to be compa-
rable to or stronger than that of other
infant neuromotor tests administered
during the first year of life.
We thank Dr Jonathan Berkowitz for his su-
perb guidance and help in data analysis and
Kate Junaid, Lynn Rogers, and Liisa Hohlsti
for their invaluable assistance in data collec-
tion. We also sincerely thank the families and
infants who took part in this study.
REFERENCES
1. Cummins SK. Prenatal and perinatal
factors and the epidemiology of cere-
bral palsy. Phys Med Rehabil 1991;5:
403-15.
2. Kirshner B, Guyatt GH. A method-
ologic framework for assessing health
indices. J Chron Dis 1985;38:27-36.
3. Bayley N. Bayley Scales of Infant De-
velopment. 2nd ed. San Antonio (TX):
Psychological Corporation; 1993.
4. Stengler SR, Huber CJ, Routh DK.
Screening growth and development of
preschool children: a guide for test se-
lection. New York: McGraw-Hill;
1980. p. 34-60.
5. Chandler LS, Andrews MS, Swanson
MW. Movement assessment of infants:
a manual. Rolling Bay (WA): Authors;
1980.
THE JOURNAL OF PEDIATRICS
VOLUME 139, NUMBER 2
6. Harris SR, Haley SM, Tada WL,
Swanson MW. Reliability of observa-
tional measures of the Movement As-
sessment of Infants. Phys Ther 1984;
64:471-7.
7. Harris SR, Swanson MW, Andrews
MS, Sells CJ, Robinson NM, Bennett
FC, et al. Predictive validity of the
Movement Assessment of Infants. J
Dev Behav Pediatr 1984;5:336-42.
8. Haley SM, Harris SR, Tada WL,
Swanson MW. Item reliability of the
Movement Assessment of Infants.
Phys Occup Ther Pediatr 1986;6:
21-39.
9. Harris SR. Early predictors of cerebral
palsy in low birthweight infants. Dev
Med Child Neurol 1987;29:508-19.
10. Harris SR. Early detection of cerebral
palsy: sensitivity and specificity of two
motor assessment tools. J Perinatol
1987;7:11-5.
11. Harris SR. Early diagnosis of spastic
diplegia, spastic hemiplegia, and quad-
riplegia. Am J Dis Child 1989;143:
1356-60.
12. Amiel-Tison C, Korobkin R, Esque-
vaucouloux M. Extensor hypertonia: a
clinical sign of insult to the central ner-
vous system of the newborn. Early
Hum Dev 1977;1/2:188-90.
13. Ellenberg J, Nelson K. Early recogni-
tion of infants at risk for cerebral palsy.
Dev Med Child Neurol 1981;23:705-16.
14. Harris SR. Movement analysis: an aid
to early diagnosis of cerebral palsy.
Phys Ther 1991;71:215-21.
15. Touwen BCL, Hadders-Algra M. Hy-
perextension of neck and trunk and
shoulder retraction in infancy: a prog-
nostic study. Neuropediatrics 1983;14:
202-5.
16. Harris SR. Development of an infant
neuromotor assessment tool. Mary E.
Switzer Research Fellowship final re-
port. Washington (DC): National In-
stitute on Disability and Rehabilitation
Research; 1991.
17. Harris SR. Parents’ and caregivers’ per-
ceptions of their children’s development.
Dev Med Child Neurol 1994;36:918-23.
18. Bayley N. Bayley Scales of Infant De-
velopment. New York: Psychological
Corporation; 1969.
19. Harris SR, Daniels LE. Content valid-
ity of the Harris Infant Neuromotor
Test. Phys Ther 1996;76:727-37.
20. Bartko J. The intraclass correlation
coefficient as a measure of reliability.
Psychol Rep 1966;19:3-11.
21. Shrout PE, Fleiss JL. Intraclass corre-
lation: uses in assessing rater reliabili-
ty. Psychol Bull 1979;86:420-8.
22. Ottenbacher KJ, Tomchek SD. Relia-
HARRIS AND DANIELS
bility analysis in therapeutic research:
practice and procedures. Am J Occup
Ther 1993;47:10-6.
23. Anastasi A. Psychological testing. New
York: MacMillan Publishing Compa-
ny; 1988.
24. Piper MC, Darrah J. Motor assess-
ment of the developing infant.
Philadelphia: WB Saunders; 1994.
25. McCarthy D. McCarthy Scales of
Children’s Abilities. San Antonio: The
Psychological Corporation; 1972.
26. Wechsler D. Wechsler Preschool and
Primary Scale of Intelligence–Revised.
San Antonio: The Psychological Cor-
poration; 1989.
27. Elliott CD. Differential ability scales:
administration and scoring manual.
San Antonio: The Psychological Cor-
poration; 1990.
28. Meyer CR. Measurement in physical
education. New York: The Ronald
Press Co; 1974. p. 86-9. Cited by: Pal-
isano RJ. Concurrent and predictive
validities of the Bayley Motor Scale and
the Peabody Developmental Motor
Scales. Phys Ther 1986;66:1714-9.
29. Sackett DL, Haynes RB, Guyatt GH,
Tugwell P. Clinical epidemiology: a
basic science for clinical medicine.
2nd ed. Boston: Little, Brown & Co;
1991.
253