European Journal of Clinical Nutrition (2008) 62, 770–780

& 2008 Nature Publishing Group All rights reserved 0954-3007/08 $30.00
www.nature.com/ejcn

ORIGINAL ARTICLE
Nutrition Screening Tool for Every Preschooler
(NutriSTEP ): validation and test–retest reliability of
TM

a parent-administered questionnaire assessing

nutrition risk of preschoolers
JA Randall Simpson1, HH Keller1, LA Rysdale2 and JE Beyers2

1
Department of Family Relations and Applied Nutrition, University of Guelph, Guelph, ON, Canada and 2Sudbury & District Health
Unit, Sudbury, ON, Canada

Objectives: Study 1: To establish the validity of scores on Nutrition Screening Tool for Every Preschooler (NutriSTEP), a
community-based parent-administered screening tool for assessing nutrition risk, by comparing scores to an expert rating. Study
2: To demonstrate test–retest reliability of NutriSTEP.
Subjects/Methods: Study 1: Parents of 269 preschoolers (of 294 parents recruited from the community), completed the
NutriSTEP questionnaire; a registered dietitian (RD) assessed the nutritional status (based on medical and nutritional history, 3
days of dietary recall and anthropometric measurements) of these preschoolers and rated their nutritional risk (1 (low) to 10
(high risk)). Receiver operating characteristic (ROC) curves were used to establish validity and determine appropriate cut points
based on sensitivity and specificity. Study 2: Parents of 140 preschoolers (of 161 recruited) completed NutriSTEP on two
occasions. Intraclass correlation (ICC) and k were used to assess reliability.
Results: Study 1: Scores on NutriSTEP and the RD rating were correlated (r ¼ 0.48, P ¼ 0.01). Area under the ROC curve for the
high risk RD rating (score 8 þ ) and the moderate risk rating (score 5 þ ) were 81.5 and 73.8%, respectively. A moderate risk cut
point of 420 and high risk cut point of 425 were identified for the NutriSTEP scores. Study 2: The NutriSTEP score was reliable
between administrations (ICC ¼ 0.89, F ¼ 16.7, Po0.001). Most items on the questionnaire had adequate (k40.5) or excellent
(k40.75) agreement.
Conclusions: The NutriSTEP questionnaire is both valid and reliable for determining nutritional risk in preschoolers.
European Journal of Clinical Nutrition (2008) 62, 770–780; doi:10.1038/sj.ejcn.1602780; published online 6 June 2007

Keywords: screening; preschoolers; nutrition risk; validity; reliability

Introduction and academic performance, social well being and future

It is well known that childhood nutrition directly affects
growth and development, health status, school readiness
Correspondence: Professor JA Randall Simpson, Department of Family
Relations and Applied Nutrition, University of Guelph, Guelph, ON N1G
2W1, Canada.
E-mail: rjanis@uoguelph.ca
Guarantor: JA Randall Simpson.
Contributors: JARS, HHK, LAR and JEB planned and designed the study and
obtained funding. LAR managed and coordinated the project. JARS
coordinated data collection for some of the test–retest reliability. HHK was
responsible for data analysis. All authors contributed to the writing and
revision of the paper.
Received 14 August 2006; revised 20 March 2007; accepted 12 April 2007;
published online 6 June 2007
contributions to society (Nicklas and Johnson, 2004). In the
long term, eating habits will also influence the occurrence of
chronic diseases such as diabetes, heart disease, obesity and
osteoporosis (Freedman et al., 1999; American Diabetes
Association, 2000; Whiting, 2002; Nicklas et al., 2003). As
eating habits and patterns are established at an early age, it
is important to intervene early if there are concerns. Pre-
schoolers, aged 3–5 years, are especially vulnerable to poor
nutrition. In North America and other developed countries,
concerns include: poor growth, iron deficiency anemia, food
allergies/intolerances, delayed/inadequate acquisition of
feeding/eating skills, unhealthy feeding/eating environ-
ments, unhealthy body image/self-esteem and food in-
security and overweight and obesity (Sanderson et al., 2001;

National Institute of Nutrition, 2002; Nicklas et al., 2003;
Polhamus et al., 2003; Bangash and Bahna, 2005; Shields,
2005; Badeyan and Guignon, 2006; Ells et al., 2006; National
Statistics Online, 2006).
Nutrition risk can be defined as the presence of character-
istics or risk factors that can lead to impaired nutritional
status (American Dietetic Association, 1994). For preschoo-
lers, nutrition risk runs the spectrum from under- to
overnutrition and occurs to those in poverty as well as those
living in relative affluence (Alaimo et al., 2001; Whitaker and
Orzol, 2006). Systematic screening systems that identify
these characteristics or risk factors before school entry may
be an efficient means of identifying children who require
services and preventive interventions (American Dietetic
Association, 1994; Whitlock et al., 2005).
To the best of our knowledge, there are no valid and
reliable preschool nutrition screening tools that can be
readily completed by parents. The PEACH survey (Parent
Eating and Nutrition Assessment for Children with Special
Needs) (Campbell and Kelsey, 1994) was developed in the
United States for use with children with developmental
delays and disabilities. Other screening tools have been
developed for clinical, in-patient hospital settings (Boutry
and Needlman, 1996; Sermet-Gaudelus et al., 2000). Two
nutrition screens produced in British Columbia, Canada
(Richards and Wilkinson, 1998; Yeung, 1998) consisted of
questionnaires that required completion and interpretation
by health professionals and have not been assessed for
reliability and validity. Pilot validation work done in the
United Kingdom (Wardle et al., 2001) has produced the
children’s eating behavior questionnaire (CEBQ). Initial
findings suggest that the CEBQ might be a useful measure
of eating style that can lead to obesity or eating disorders,
although the risk factors included are limited. However,
none of these tools are suitable for use in the general
preschool population to measure the overall construct of
nutritional risk. Nutrition Screening Tool for Every Pre-
schooler (NutriSTEP) is a community-based, parent-adminis-
tered nutrition screening tool that has been developed by
the authors.
The purpose of this study was to identify criterion validity
of NutriSTEP by comparing this simple parent-administered
questionnaire to the expert rating of nutrition risk by a
registered dietitian (RD) (Study 1: criterion validation). The
second purpose of this paper was to demonstrate test–retest
reliability of the parent-completed NutriSTEP (Study 2: test–
retest reliability).
Subjects and methods
Development of NutriSTEP (Nutrition Screening Tool for Every
Preschooler)
NutriSTEP is a community-based, parent-administered nutri-
tion screening tool that includes risk factors that have been
validated for content by parents and professionals. The
Validation and reliability of NutriSTEPt
JA Randall Simpson et al
771
development of NutriSTEP, with the involvement of almost
2000 preschoolers and their parents and more than 50
multisectoral partners, is shown in Figure 1. Initial steps in
the 7-year development process (following the methodolo-
gical template of Keller et al., 2000) included identifying the
construct of nutrition risk for preschoolers. The constructs of
nutrition risk were defined as: physical growth, food and
fluid intake, physical activity and sedentary behavior and
factors affecting food intake for this age group (e.g., food
security, feeding environment). NutriSTEP development, in
both English and French, has been inclusive, with cultural
consideration in examples given for foods and in termino-
logy used. Some of the cultural groups from across Canada
included in the development were: First Nations, Chinese,
Middle Eastern and European. The version of NutriSTEP used
in this validation study was extensively refined and then
pilot tested with 80 preschoolers to ensure that it could be
readily completed by parents with minimal administrative
guidance. NutriSTEP includes 17 items: five questions are
focused on food group intake while the remaining 12 cover
the aspects of the above-noted nutrition risk constructs (see
Appendix A for item stems). Each question has two to five
response options. Responses range in score from 0 (no risk)
to 4 (risk) and question responses are summed to provide an
index where an increased score indicates increased nutrition
risk. The maximum score is 68.
Subjects
For Studies 1 and 2, participants were recruited by study
personnel from several community programs including
child-care centers and other programs for preschoolers
(e.g., Ontario Early Years Centers) in both Southern and
Northern Ontario, including rural and urban settings. The
convenience samples for each study were recruited indepen-
dently and were diverse, including potentially high-risk
families. Inclusion criteria included: being the parent/
primary caregiver of a preschool child (3–5 years); having
lived in Canada for at least 5 years; and, being able to read
and write either English or French at a grade six level. The
studies were approved by the Research Ethics Board at the
University of Guelph and participating agencies; parents
provided informed written consent before data collection.
Incentives included a NutriSTEP measuring cup, two parent-
education booklets produced by the NutriSTEP team (Sud-
bury & District Health Unit, 2004a, b) and $20 (Canadian)
(for Study 1 only).
Study 1: criterion validation. A sample size of B300 was
estimated to be sufficient based on previous validation work
and methodology (Streiner and Norman, 1996; Keller et al.,
2001; Jones, 2004a). Two hundred and ninety-four subjects
were enrolled between May 2005 and January 2006; 269 had
complete data for validation analyses. The parents/guardians
who initially provided consent and then did not complete
the study in full were no different from those included in the
European Journal of Clinical Nutrition
 Validation and reliability of NutriSTEPt
JA Randall Simpson et al
772
analyses, with the exception that a higher proportion of
non-Canadian-born mothers did not complete the entire
protocol (22 vs 11% of Canadian born; w2 ¼ 5.5, P ¼ 0.049).

Study 2: test–retest. A sample size of B150 was estimated to

be sufficient for test–retest reliability (Streiner and Norman,
1996; Jones, 2004b). One hundred and sixty-one subjects
were enrolled between October 2005 and February 2006;
complete and useable data were available for 140. Those who
did not complete the second questionnaire were more likely
to have only a high school education (26 vs 11%; w2 ¼ 11.65,
P ¼ 0.02) and to have a younger second child (2.272.4 vs
3.872.4 years; t ¼ 2.3, P ¼ 0.04) than those who completed
both questionnaires.

Procedures
Study 1: criterion validation. The criterion for comparison of
the questionnaire was a nutritional assessment completed by
one of three specially trained dietitians. Parents filled out the
17-item NutriSTEP and a standardized demographic ques-
tionnaire adapted from Statistics Canada (Statistics Canada,
2001); these were placed in sealed envelopes so that the
dietitian remained blinded to the results of the NutriSTEP
questionnaire. Parents were also given instructions, forms
and a measuring cup to complete a 3-day food record that
included two weekdays and one weekend day. Weight and
height of the children were measured in triplicate using a
calibrated scale (Lifesource MD, Rosscraft, CA, USA) accurate
to 50 g and a portable stadiometer (Road Rod, 214, SECA,
USA; inter-rater reliability among dietitians, coefficient of
variation o1%) accurate to 0.1 cm, respectively. The dieti-
tian completed a brief clinical assessment identifying any
physical signs of malnutrition in the child at the first visit
(weight, anemia, failure to thrive, etc).
At a subsequent visit (within 1 month), the dietitian
reviewed the details of the 3-day record with the parent
(comparison to Canada’s Food Guide to Healthy Eating
(CFGHE) Focus on Preschoolers (Health Canada, 1995)) and
using this evaluation as a basis, completed a clinical history
including questions on health, diet, eating behavior prac-
tices of the child and family, physical activity, television use,
etc. The dietitians used a standardized risk-rating guide
(Appendix B) including objective and subjective information
to rate nutrition risk in these preschoolers. This guide was
developed from literature and underwent content validation
by an external expert review group. The risk rating was based
on a 10-point scale (1–4 ¼ low risk, 5–7 ¼ moderate risk,
8–10 ¼ high risk) adapted from previous validation work
(Keller et al., 2001; Keller, 2005). Where significant nutrition
problems were evident, the dietitian referred the family to
local resources.

Study 2: test–retest reliability. At the point of recruitment,

parents filled out a general demographic questionnaire and
Figure 1 Development of NutriSTEP – critical path. the NutriSTEP. At a subsequent visit (2–4 weeks later and
either at the recruitment setting or home), a second

European Journal of Clinical Nutrition

 Validation and reliability of NutriSTEPt
JA Randall Simpson et al
773
NutriSTEP questionnaire was completed by the same parent/ Table 1 Studies 1 and 2 parent respondent and index child
guardian. characteristics

Parent characteristic Study 1 (n ¼ 269) Study 2 (n ¼ 140)

mean7s.d. mean7s.d.
Statistical analyses
Data entry was done in duplicate in EpiInfo version 6 Respondent age (years) 34.2 (5.6) 35.0 (6.3)
(Centers for Disease Control, Atlanta, GA, USA) to ensure Number in household 4.4 (1.3) 4.1 (0.93)
Number of children 2.3 (1.2) 2.1 (0.77)
accuracy. Basic descriptive analyses were completed for all
samples and subgroups for comparison purposes on demo- % %
graphics and health information. Receiver operating char-
Gender
acteristic (ROC) curves are the preferred methodology
Female 93.7 93.6
(Streiner and Norman, 1996; Green and Watson, 2006) for Male 6.3 6.4
determining validity and developing cut points for deter-
mining risk. NutriSTEP was compared to the dietitian’s risk Marital status
Married 88.7 92.9
rating (1–10 rating) by the use of ROC curves. Using
Single 6.4 1.4
moderate (5 þ ) and high risk (8 þ ) cut points on this Separated/divorced/widowed 4.9 5.7
10-point dietitian rating scale, ROC curves were created. A
high area under the curve (AUC) indicates that the measured Education
oHigh school 11.6 —
variable is consistent with scoring of the criterion (risk/no
Graduated high school 5.3 10.7
risk) (Streiner and Norman, 1996). Appropriate risk cut Some post-secondary 14.0 8.6
points on NutriSTEP were identified by comparing trade-offs Graduate post-secondary 69.1 80.6
of sensitivity (SN) and specificity (SP) for various scores based
Income
on the ROC.
o30 000 12.8 7.9
Test–retest of total NutriSTEP scores was assessed by 30–59 000 21.1 22.9
intraclass correlation (ICC). Reliability of individual ques- 60 000 þ 59.9 65.0
tionnaire items was assessed by using k when the item scores Don’t know 6.2 2.1
were dichotomized at meaningfully important cut points (as
Mother’s language
determined by consensus of authors) indicating increased English 57.7 82.9
risk. A k above 0.5 is considered adequate with scores greater French 8.5 10.0
than 0.75 indicating excellent agreement (Streiner and Mother immigrated 34.5 7.1
Father immigrated 37.6 13.6
Norman, 1996). SPSS version 12.2 (SPSS Inc., Chicago, IL,
USA) and SAS version 8.2 (SAS Institute Inc., Cary, NC, USA) Index child Mean7s.d. Mean7s.d.
were used for all analyses.
Weight (kg) 17.6 (3.3) N/A
Height (cm) 103.1 (8.1) N/A
Body mass index 16.5 (1.8) N/A
Results
% %
Table 1 provides sample descriptions for the two samples; the
Medical condition 15.0 14.6
genders of the children were equally represented, but ages of
these children were predominately 3 and 4 years. A little over First language
one-third of mothers and fathers in Study 1 (validation) were English 69.7 85.0
French 7.0 7.1
immigrants to Canada and almost 20% of validation sample
Born outside of Canada 19.4 4.3
children were born outside of Canada. Study 1 and 2 parents
were relatively similar on other characteristics: in their mid- Age (years)
30s, married, having two children, with the majority having 3 49.3 51.7
4 31.3 24.4
a post-secondary education. However, about 12% of the
5 19.4 23.7
Study 1 parent respondents had less than a high school
education and almost 13% had a household income less Gender
than $30 000 (Canadian). As the Study 1 sample was more Female 50.0 40.0
Male 50.0 60.0
diverse, there were relatively high proportions of mothers
and sample children who did not speak either English or
French as their first language (33.9 and 24.6%, respectively).
In Study 1, the average dietitian risk rating was 3.9
(s.d. ¼ 1.9); 62.8% of preschoolers were low risk (score 1–4), Table 2 provides reliability results for individual NutriSTEP
32.3% were moderate risk (score 5–7) and 4.8% were items as well as an indication of the proportion of the
considered high risk (score X8). samples at risk for specific index items. Samples had a high

European Journal of Clinical Nutrition

 Validation and reliability of NutriSTEPt
JA Randall Simpson et al
774
level of risk (30% or more of sample was at risk) for the Table 2 Proportion of respondents with individual NutriSTEP items
following NutriSTEP items: grain products (p3 servings per indicating risk for Studies 1 and 2 and k statistic indicating item reliability
day), fruit (p2 servings per day), vegetables (p1 serving NutriSTEP item Study 1 Study 2 ka
per day), meat and alternatives (only for Study 1; p1 serving (n ¼ 269)% (n ¼ 140)%
per day), the child did not control amount of food consumed
(sometimes to never), use of vitamin supplements (only for Grains
43  per dayb 45.7 50.0 0.56
Study 1; most of the time to always) and participating in
p3  per dayc 54.3 50.0
sedentary activities (only for Study 1; 3 or more hours per
day, sits, watches television, etc). In keeping with the Milk/substitute products
diversity of the validation sample, higher proportions of 43 per dayb 75.5 85.0 0.39
children exhibited risk as compared to the test–retest sample p2 per dayc 24.5 15.0
for: milk/substitute product intake (24.5 vs 15.0%), low
Fruit
vegetable consumption (44.6 vs 34.3%), low meat and 42  per dayb 45.4 46.4 0.52
alternatives (31.6 vs 22.1%), frequent use of fast food (12.6 p2  per dayc 54.6 53.7
vs 7.9%), frequent use of vitamin supplements (31.6 vs
Vegetables
22.9%) and frequent sedentary activities (32.7 vs 21.4%).
41 per dayb 55.4 65.7 0.71
Participants in the test–retest sample were more likely to p1 per dayc 44.6 34.3
indicate that the child consumed food two or fewer times per
day (18.6%) than in the validation sample (3%). Meat and alternatives
41 per dayb 68.4 77.9 0.55
p1 per dayc 31.6 22.1

Study 1: validity of NutriSTEP Fast food

NutriSTEP scores ranged from 4 to 46 (of a possible 68) with a o2  per weekb 87.4 92.1 0.76
median of 18.0 (s.e.m. ¼ 0.42). When compared with the X2  per weekc 12.6 7.9
three levels (low, moderate and high risk) of dietitian risk
Food is expensive
score, there was a significant difference in NutriSTEP scores Never to rarelyb 83.3 82.9 0.70
(F ¼ 37.2, Po0.0001). For those children whom the dietitian Sometimes to alwaysc 16.7 17.1
indicated as low risk, the mean score was 16.1 (s.d. ¼ 5.7), for
moderate risk, the mean score was 21.1 (s.d. ¼ 6.4) and for Chewing/swallowing difficulty
Never to rarelyb 90.7 90.7 0.64
high risk, the mean score was 28.5 (s.d. ¼ 9.6). As higher risk
Sometimes to alwaysc 9.3 9.3
occurs with a higher score, this indicates that NutriSTEP
scores were reflective of the dietitian rating. Additionally, the Not hungry at meals due to drinking
Spearman’s rho between the NutriSTEP score and the Never to rarelyb 75.5 77.9 0.56
Sometimes to alwaysc 24.5 22.1
dietitian rating was 0.48 (P ¼ 0.01).
Figure 2 provides the ROC curve for NutriSTEP when Eating frequency
compared to the high risk dietitian rating (score 8 þ ). The 43  per dayb 97.0 81.4 0.73
AUC for this ROC curve was 81.5%. For the moderate risk p2  per dayc 3.0 18.6
dietitian rating cut point (score 5 þ ) (ROC not shown), the
Child controls amount consumed
AUC was 73.8%. Table 3 provides the SN and SP for Always/most of the timeb 62.8 68.6 0.56
NutriSTEP scores for both dietitian rating cut-points as well Sometimes to neverc 37.2 31.4
as prevalence of preschoolers with this score or higher. Based
Watches television at meals
on these data, two cut points have been chosen to classify
Never to sometimesb 89.2 92.1 0.57
preschoolers into one of three risk categories based on the Most of the time to alwaysc 10.8 7.9
NutriSTEP score: score p20 ¼ low risk; score 420 and
p25 ¼ moderate risk and score 425 ¼ high risk. Based on Uses vitamin/mineral supplements
Never to sometimesb 68.4 67.1 0.81
these data, a cut point of 420 has a SN of 53–69% and SP of
Most of the time to alwaysc 31.6 22.9
79–69% as compared to dietitian rating of moderate and
high risk, respectively. A cut point of 425 has a SN of 84– Physically active
92% and a SP of 46–36% as compared to dietitian rating of Enoughb 82.5 78.6 0.76
Needs morec 17.5 21.4
moderate and high risk, respectively.
Sedentary activity during day
o3 hb 67.3 78.6 0.71
Study 2: test–retest reliability 43 hoursc 32.7 21.4
The NutriSTEP overall score was reliable between adminis-
Growth adequate
trations by parents (ICC ¼ 0.89; 95% confidence interval Yesb 91.1 97.9 1.0
0.85, 0.92; F ¼ 16.7; Po0.001). Table 2 outlines the k Noc 8.9 2.1

European Journal of Clinical Nutrition

 Validation and reliability of NutriSTEPt
JA Randall Simpson et al
775
Table 2 Continued NutriSTEP has been carried out using well-accepted proce-
NutriSTEP item Study 1 Study 2 ka dures and methods as described by other authors (Streiner
(n ¼ 269)% (n ¼ 140)% and Norman, 1996; Keller et al., 2000; Jones, 2004a, b, c).
The crucial need for such a tool is demonstrated by the
Weight prevalence of risk in these convenience samples. More than
Appropriateb 81.8 90.7 0.76
Too much/littlec 18.2 9.3
one-third of the validation sample of preschoolers was
considered by the RDs to be at moderate or high risk for
a
Reliability k coefficient for Study 2. nutrition problems. If the suggested moderate risk cut point
b
Indicates no risk. of 420 for NutriSTEP is used, the prevalence of nutrition risk
c
Indicates risk.
was 23.5 and 33.2% in the test–retest and validation samples,
respectively. There are currently no other data with estimates
of nutrition risk in this age group. It is, however, known that
the prevalence of nutritional problems such as overweight
and obesity in this age group is 26.5% in the United States
(Ogden et al., 2006) and 21% in Canada (Shields, 2005) and
is increasing in the United Kingdom (National Statistics
Online, 2006) and in France (Badeyan and Guignon, 2006);
in the validation sample, the prevalence of overweight and
obesity was 21%.
NutriSTEP also identified a worrisome prevalence of
children at potential risk for low frequency of intake of
major food groups relative to recommendations in CFGHE’s
Focus on Preschoolers (Health Canada, 1995). Data with
which to compare our results are limited; however, a recent
study of dietary intakes at child-care centers in the US found
that children were not meeting the recommendations in the
Food Pyramid Guide for Young Children (United States
Department of Agriculture, 1999; Padget and Briley, 2005).
As it is recommended that foods, rather than supplements,
should be emphasized as optimal sources of nutrition
(Kleinman, 2002; Briefel et al., 2006), the regular use of
dietary supplements by 32% of the sample is of concern. The
use of vitamin and mineral supplements in a 1-month period
in 2004 in preschoolers in Canada has recently been reported
Figure 2 Receiver operator characteristic (ROC) curve for score on to range from 9 to 12% (Statistics Canada, 2006), whereas the
NutriSTEP compared to registered dietitian risk rating of nutritional prevalence of dietary supplement use (at least one dietary
risk (risk X 8 on rating of 1 to 10). supplement in the previous month) in preschool children in
the United States has been reported to be 40% (NHANES
1999–2000) (Ervin et al., 1999; Briefel and Johnson, 2004).
statistics for the dichotomized index items. k ranged from One of the NutriSTEP items was designed to determine
0.39 (low milk/substitute product intake) to 1.0 (parent’s parental control over feeding. Thirty-seven percent of
perception of growth). Other than the milk/substitute parents reported that they often do not let their children
product question, all items had adequate (k40.5) or decide how much to eat. There is an accumulating body of
excellent (k40.75) agreement. evidence to suggest that strict parental control over how
much a child eats, particularly food restriction, may play a
role in the development of childhood overweight/obesity
Discussion (Johnson and Birch, 1994; Fisher and Birch, 1999; Birch and
Fisher, 2000). Also of great concern is the high proportion
The major finding of this research is that the NutriSTEP (33%) of preschool children whose sedentary activities, such
questionnaire has been demonstrated to be valid, based as television viewing, exceeded 3 hours a day. Excess
on the best alternative to a ‘gold standard’, the clinical sedentary activity, particularly in the form of TV viewing
assessment of children by RDs. To the best of our knowledge, presents the potential for nutritional risk, especially in
this research is the first to describe the validation and young children (Veugelers and Fitzgerald, 2005).
reliability testing of a nutrition risk screening questionnaire The high AUC for the ROC curves indicates the validity of
that can be completed by parents/caregivers of preschoolers the NutriSTEP questionnaire. Two cut points of 420 and
in a community-based setting. The extensive development of 425 are recommended for use. The SN of these cut points

European Journal of Clinical Nutrition

 Validation and reliability of NutriSTEPt
JA Randall Simpson et al
776
Table 3 Cumulative prevalence of participants at/above NutriSTEP drinking) that had fair reliability (ko0.60) demonstrated less
score and comparison of SN and SP using the moderate and high risk risk on second administration, suggesting a learning effect
dietitian cut points as a comparison for individual NutriSTEP score values
by parents (Streiner and Norman, 1996). Nevertheless, the
NutriSTEP Cumulative prevalence of Moderate High reliability was fair to excellent for all but one item.
score participants at/above risk X5 risk X8 Comparisons of test–retest reliability of NutriSTEP with
NutriSTEP score other nutrition screening questionnaires for children are
Study 1 (%) Study 2 (%) SN (%) SP (%) SN (%) SP (%) not possible as reliability testing for the CEBQ (Wardle et al.,
2001) was assessed using Pearson’s correlation and data are
4 100 100 1 100 8 100 not available for the PEACH questionnaire (Campbell and
5 99.5 100 2 100 15 100 Kelsey, 1994). The test–retest reliability of NutriSTEP, how-
6 99.1 98.5 3 100 15 100
7 98.7 97.1 5 100 15 99 ever, compares favorably with tools developed for other
8 97.6 95.7 6 100 15 98 populations (Laporte et al., 2001; Keller, 2005).
9 96.1 91.4 9 99 23 97 The next step in the development of NutriSTEP is to design
10 92.4 87.1 11 99 31 96 and evaluate models for its ethical implementation (Rush,
11 88.7 82.1 13 99 31 96
12 84.6 75.0 17 98 46 95 1997; Keller et al., 2006). This involves targeting preschool
13 79.0 70.0 22 97 69 93 children in need of screening and identifying nutrition
14 73.4 67.9 25 96 69 91 problems and appropriate courses of action (e.g., assessment,
15 67.8 62.9 26 94 69 89 resources), identifying a referral/resource framework to meet
16 65.2 55.0 26 93 69 89
17 59.6 50.0 32 92 69 86 needs and following the client after screening (Keller et al.,
18 52.2 41.4 37 91 69 83 2006). At this point, it is recommended that a cut point of
19 45.1 35.0 42 87 69 79 420 for NutriSTEP be used to identify risk that can be met
20 38.4 26.4 46 84 69 75 with public health services and interventions. A higher cut
21 33.2 23.5 53 79 69 69
22 27.3 17.8 58 73 77 64 point of 425 indicates high risk and these children should
23 24.0 14.9 65 67 77 57 be referred to a dietitian for further assessment and
24 19.5 11.3 72 60 77 49 treatment. The prevalence of this high risk in the validation
25 16.9 6.3 80 53 92 42 (13.9%) and test–retest (6.3%) samples is lower and is more
26 13.9 6.3 84 46 92 36
27 13.5 4.9 87 44 100 34 realistic for referral to a dietitian.
28 11.6 4.9 90 37 100 28 Strengths of this research are numerous. NutriSTEP ques-
29 10.1 4.9 93 30 100 22 tions have undergone extensive development and refine-
30 7.5 4.9 96 22 100 16 ment. The diverse nature of the population of preschoolers,
31 5.6 4.9 96 16 100 12
32 4.9 3.5 98 11 100 8 with whom we have developed NutriSTEP, and the con-
33 3.8 3.5 100 7 100 4 current development in a language other than English,
35 2.3 1.4 100 4 100 3 makes this tool applicable for use across Canada and in
36 1.9 1.4 100 2 100 2 similar populations in other locales such as the United
39 1.2 0.7 100 2 100 1
44 0.8 0.7 100 1 100 1 States, the United Kingdom and Europe. The applicability of
47 0.4 0.7 100 0 100 0 NutriSTEP for use in a US population is currently being
studied at Northern Illinois University (Henry B, personal
Abbreviation: SN, sensitivity; SP, specificity. communication, 2006).
Limitations of this research include the lack of a true gold
standard for validation as nutritional assessment is a
subjective, rather than an objective process. By creating a
compares favorably with the PEACH survey (SN 88.6%) that framework for assessment that was followed by all of the
was developed in the United States for use in children from dietitians, differences were minimized. This framework was
birth to 5 years who have or who are at risk of developmental also validated for content by pediatric clinical nutrition
problems (Campbell and Kelsey, 1994). The relatively good experts. A further criticism of this work may include the use
SN and SP of this index are likely to be a result of the careful of three RDs to assess the children rather than one, thus
and comprehensive developmental process for NutriSTEP. introducing inter-rater variation in risk ratings; however,
Without including objective measures such as measured 80% of the dietitian assessments were done by one dietitian.
height or weight, higher SN for questionnaire-based items is Additionally, the assessment did not include biochemical
difficult to attain. Other multiple-response option self- measurements as they were not considered to be feasible
administered questionnaires have a similar AUC, SN and SP and would have influenced participation rate. Other than
(Keller et al., 2001; Keller, 2005). identifying specific nutrient deficiencies, biochemistry
Individual NutriSTEP items and the total score are reliable would not have provided additional data to support clinical
when administered to parents on two separate occasions. It judgment of overall risk. Finally, some data were unfortu-
was found that responses to some questions (consumption of nately missing. As with any community-based research,
grain products and fruit and not hungry at meals due to this can influence external validity. Comparisons between

European Journal of Clinical Nutrition


dropouts and participants indicated minimal differences and
we do not believe that these missing data influenced the
validity or reliability of NutriSTEP.
In conclusion, we have demonstrated that the score on
NutriSTEP is valid and reliable and fills the need for a simple
index for nutrition screening in the preschool population.
Beyond the determination of nutritional risk, NutriSTEP,
with its accompanying nutrition-education resources
(Sudbury & District Health Unit, 2004a, b), can increase
food and nutrition awareness among parents and caregivers
and health, education and social service providers. Nutri-
STEP offers insight into the gamut of community-level
preschool nutrition issues, thereby allowing for population-
based health planning and the development of appropriate
programs, services and staffing models to meet these
identified needs. As the tool is self-administered by parents
or guardians, there is the potential for universal access. It is
feasible that NutriSTEP can be implemented for use in a
variety of settings, similar to those accessed for the valida-
tion study, in Canada and in other locales to identify those
preschoolers for whom appropriate interventions can be
recommended.
Acknowledgements
This work was supported by the Canadian Institutes of
Health Research (Institute of Population and Public Health
and Institute of Nutrition, Metabolism and Diabetes), the
Provincial (Ontario) Public Health Research, Education and
Development (PHRED) Program, the City of Greater Sudbury
Ontario Early Years Challenge Fund and Health Canada
(Population Health Fund). We especially thank the parents
and the preschoolers who participated in this study. We
thank Sandra Juutilainen, Lorianne Maclean and Suzanne
Lamoureux as validation Registered Dietitians for this
project. We acknowledge the support with recruitment of
participants from the following: Mary Turfryer from the York
Region Health Unit, Nancy Summers from the Middlesex
London Health Unit, Sharon Dinsmore from the Grey-Bruce
Health Unit, Elizabeth Shaver Heeney from Hamilton Public
Health and Community Services, Pat Hanly from the Perth
District Health Unit and Joy Walker from the Guelph
Community Health Center. We thank the following students
for their role in data collection for the test–retest reliability:
Amanda Degen, Jill Stroud and Kathleen Hennessy-Priest.
For data entry and management, we thank Jill Stroud,
Kathleen Hennessy-Priest and Alissa Palangio. We are grate-
ful to Richard Goy for statistical analysis.
References
Alaimo K, Olson CM, Frongillo Jr EA, Briefel RR (2001). Food
insufficiency, family income, and health in US preschool and
school-aged children. Am J Public Health 91, 781–786.
Validation and reliability of NutriSTEPt
JA Randall Simpson et al
777
American Diabetes Association (2000). Type 2 diabetes in children
and adolescents. Diabetes Care 23, 381–389.
American Dietetic Association (1994). ADA’s definitions for nutrition
screening and nutrition assessment. J Am Diet Assoc 94, 838–839.
Badeyan G, Guignon N (2006). Obesity and Asthma: Two Pathologies
whose Prevalence among Children is Growing, Surveyed through School
Medical Examinations. Ministry of Employment and Social Affairs
and Ministry of Health, Family Matters and the Handicapped:
Paris, France. Available at: http://www.insee.fr/en/ffc/docs_ffc/
DONSOC02g.PDF.
Bangash SA, Bahna SL (2005). Pediatric food allergy update. Curr
Allergy Asthma Rep 5, 437–444.
Birch LL, Fisher JO (2000). Mothers’ child-feeding practices influence
daughters’ eating and weight. Am J Clin Nutr 71, 1054–1061.
Boutry M, Needlman R (1996). Use of diet history in the screening of
iron deficiency. Pediatrics 98, 1138–1142.
Briefel R, Hanson C, Fox MK, Novak T, Ziegler P (2006). Feeding
infants and toddlers study: do vitamin and mineral supplements
contribute to nutrient adequacy or excess among US infants and
toddlers. J Am Diet Assoc 106, S52–S65.
Briefel RR, Johnson CL (2004). Secular trends in dietary intake in the
United States. Annu Rev Nutr 24, 401–431.
Campbell MK, Kelsey KS (1994). The PEACH survey: a nutrition-
screening tool for use in early intervention programs. J Am Diet
Assoc 94, 1156–1158.
Ells LJ, Hillier FC, Summerbell CD (2006). A Systematic Review of the
Effect of Nutrition, Diet and Dietary Change on Learning, Education
and Performance of Children of Relevance to UK Schools. University of
Teesside, School of Health & Social Care, Center for Food, Physical
Activity and Obesity Research. Available at: http://www.food.
gov.uk/multimedia/pdfs/systemreview.pdf
Ervin RB, Wright JD, Kennedy-Stephenson J (1999). Use of dietary
supplements in the United States 1988–94. Vital Health Stat 11
244, 1–14.
Fisher JO, Birch LL (1999). Restricting access to palatable foods affects
children’s behavioral response, food selection and intake. Am J
Clin Nutr 69, 1264–1272.
Freedman DS, Dietz WH, Srinivasan SR, Berenson GS (1999). The
relation of overweight to cardiovascular risk factors among
children and adolescents: the Bogalusa Heart Study. Pediatrics
103, 1175–1182.
Green SM, Watson R (2006). Nutritional screening and assessment
tools for older adults: literature review. Issues and Innovations in
Nursing Practice 54, 477–490.
Health Canada (1995). Canada’s Food Guide to Healthy Eating. Focus on
Preschoolers. Health Canada. \{H39–308/1995E\} Minister of
National Health and Welfare: Ottawa, ON, Canada. Available at:
http://www.hc-sc.gc.ca/fn-an/food-guide-aliment/res/fg_preschoolers-
prescolaire_ga_e.html.
Johnson SL, Birch LL (1994). Parents’ and children’s adiposity and
eating style. Pediatrics 94, 653–661.
Jones JM (2004a). Validity of nutritional screening and assessment
tools. Nutrition 20, 312–317.
Jones JM (2004b). Reliability of nutritional screening and assessment
tools. Nutrition 20, 307–311.
Jones JM (2004c). Development of a nutritional screening or assess-
ment tool using a multivariate technique. Nutrition 20, 298–306.
Keller HH (2005). Validity and reliability of SCREEN II (seniors in the
community: risk evaluation for eating and nutrition – version II).
Eur J Clin Nutr 59, 1149–1157.
Keller HH, Brockest B, Haresign H (2006). Building capacity for
nutrition screening. Nutr Today 41, 164–170.
Keller HH, Hedley MR, Wong Brownlee S (2000). The development of
seniors in the community: risk evaluation for eating and nutrition
(SCREEN). Can J Diet Prac Res 61, 67–72.
Keller HH, McKenzie JD, Goy R (2001). Construct validation and test-
retest reliability of SCREEN (seniors in the community: risk
evaluation for eating and nutrition). J Gerontol: Medical Science
6A, M552–M558.
European Journal of Clinical Nutrition
 Validation and reliability of NutriSTEPt
JA Randall Simpson et al
778
Kleinman RE (2002). Current approaches to standards of care for
children: how does the pediatric community currently approach
the issue? Nutr Today 37, 177–179.
Laporte M, Villalon L, Thibodeau J, Payette H (2001). Validity and
reliability of simple nutrition screening tools adapted to the
elderly population in healthcare facilities. J Nutr Health Aging 5,
292–294.
National Institute of Nutrition (2002). Iron for Health – for all Ages.
National Institute of Nutrition: Ottawa, ON, Canada.
National Statistics Online (2006). Diet & Nutrition. Proportion of
Overweight Children up. National Statistics Office: Newport, UK.
Available at: http://www.statistics.gov.uk/cci/nugget_print.asp?
ID ¼ 71.
Nicklas T, Johnson R (2004). Position of the American Dietetic
Association: dietary guidance for healthy children ages 2 to 11
years. J Am Diet Assoc 104, 660–677.
Nicklas TA, Yang SJ, Baranowski T, Zakeri I, Berenson G (2003). Eating
patterns and obesity in children. The Bogalusa Heart Study. Am J
Prev Med 25, 9–16.
Ogden CL, Carroll MD, Curtin LR, McDowell MA, Tabak CJ, Flegal
KM (2006). Prevalence of overweight and obesity in the United
States, 1999–2004. JAMA 295, 1549–1555.
Padget A, Briley M (2005). Dietary intakes at child-care centers in
central Texas fail to meet food guide pyramid recommendations.
J Am Diet Assoc 105, 790–793.
Polhamus B, Dalanius K, Thompson D, Scanlon K, Borland E, Smith B
et al. (2003). Pediatric Nutrition Surveillance 2001 Report. Depart-
ment of Health and Human Services, Centers for Disease Control
and Prevention: Atlanta, USA.
Richards C, Wilkinson S (1998). Kindergarten Nutrition Screening
Program, Community Health Services, Prevention. Okanagan Similk-
ameen Health Region: Kelowna, BC, Canada.
Rush D (1997). Nutrition screening in old people: its place in the
coherent practice of preventive health care. Annu Rev Nutr 17,
101–125.
Sanderson R, Cullis H, Rysdale L, Beyers J (2001). Community-based
pre-school nutrition screening: feasibility assessment. Public
Health & Epidemiology Report Ontario (PHERO) 12, 82–88.
Sermet-Gaudelus I, Poisson-Salomon A-S, Colomb V, Brusset M-C,
Mosser F, Berrier F. et al. (2000). Simple pediatric nutritional risk
score to identify children at risk of malnutrition. Am J Clin Nutr 72,
64–70.
Shields M (2005). Nutrition: Findings from the Canadian Community
Health Survey. Measured obesity: Overweight Canadian Children and
Appendix A
Item stems from NutriSTEP questionnaire
1. My child usually eats grain products:
2. My child usually has milk products:
3. My child usually eats fruit:
4. My child usually eats vegetables:
5. My child usually eats meat, fish, poultry or alternatives:
6. My child usually eats ‘fast food’:
7. I have difficulty buying food to feed my child because
food is expensive:
8. My child has problems chewing, swallowing, gagging or
choking when eating:
9. My child is not hungry at mealtimes because he/she
drinks all day:
European Journal of Clinical Nutrition
Adolescents. Statistics Canada: Ottawa, ON, Canada. Available
at: http://www.statcan.ca/english/research/82-620-MIE/2005001/
pdf/cobesity.pdf
Statistics Canada (2001). Canadian Community Health Survey (CCHS).
Questionnaire for Cycle 1.1 (September 2000–November 2001).
Statistics Canada: Ottawa, ON, Canada.
Statistics Canada (2006). Nutrition: Findings form the Canadian
Community Health Survey – 1994. Statistics Canada: Ottawa, ON,
Canada. Available at: http://www.statcan.ca/english/research/82-
620-MIE/2006002/tables.htm
Streiner DL, Norman GR (1996). Health Measurement Scales: A
Practical Guide to their Development and Use. Oxford University
Press: Toronto, ON, Canada.
Sudbury & District Health Unit (2004a). How to Build a Healthy
Preschooler. Sudbury & District Health Unit: Sudbury, ON,
Canada. Available at: http://www.beststart.org/resources/nutrition/
index.html.
Sudbury & District Health Unit (2004b). The ABC’s of Feeding
Preschoolers. Sudbury & District Health Unit: Sudbury, ON,
Canada. Available at: http://www.beststart.org/resources/nutrition/
index.html.
United States Department of Agriculture and Center for Nutrition
Policy and Promotion (1999). Food Guide Pyramid for Young
Children. United States Department of Agriculture. Available at:
http://www.usda.gov/cnpp/KidsPyra/.
Veugelers PJ, Fitzgerald AL (2005). Prevalence of and risk factors for
childhood overweight and obesity. Can Med Assoc J 173, 607–613.
Wardle J, Guthrie CA, Sanderson S, Rapoport L (2001). Development
of the children’s eating behaviour questionnaire. J Child Psychol
Psychiatry 42, 963–970.
Whitaker RC, Orzol SM (2006). Obesity among US urban preschool
children: relationships to race, ethnicity, and socioeconomic
status. Arch Pediatr Adolesc Med 160, 578–584.
Whiting SJ (2002). Obesity is not protective for bones in childhood
and adolescence. Nutr Rev 60, 27–30.
Whitlock EP, Williams SB, Gold R, Smith PR, Shipman SA (2005).
Screening and interventions for childhood overweight: a summary
of evidence for the US Preventive Services Task Force. Pediatrics
116, e125–e144.
Yeung H (1998). Development of a Nutrition Screening Tool to Identify
Preschool-aged Children at Risk for Nutrition Problems: Determination
of Nutritional Concerns of Preschool-aged Children of Southeast
Vancouver. Vancouver/Richmond Health Board: Vancouver, BC,
Canada.
10. My child usually eats: (number of times per day)
11. I let my child decide how much to eat:
12. My child eats meals while watching TV:
13. My child usually takes supplements:
14. My child: (gets enough/needs more physical activity)
15. My child usually watches TV, uses the computer, and
plays video games:
16. I am comfortable with how my child is growing:
17. My child: (weighs too little/much)
Each question has a minimum score of 0 (no risk) to 4
(risk) with the potential of two to five frequency response
options. The question responses are summed to provide an
index where an increased score indicates increased nutrition
risk; the maximum score is 68.

Appendix B
Abbreviated standardized criteria for nutritional risk rating used
for NutriSTEP validation
Table B1
Low risk
Growth, physical and health indicators
Weight 5–95th percentile
Height 5–95th percentile
BMI for age 5–85th percentile
Healthy
No dental problems
No GI problems
Food intake, restrictions and supplements
Usually consumes most food groups
at or 4than minimum daily
recommended servings and sizes
Usually eats at regular times
throughout the day (breakfast,
lunch, supper)
Usually offered snacks 2–3 times a
day
Eats a variety of foods
Drinks at least 2 cups of milk daily
Drinks less than 6 oz (3/4 cup) juice
daily
Eats fast foods a few times a month
or less
No food allergy, restriction or diet
concerns
No supplement concerns or issues
Good appetite and intake most days
of the week
Diet and feeding related factors including age appropriate behaviors
Occasional/few food restrictions/
preferences (food jags) or
sometimes considered a ‘picky
eater’ by parent
Occasional mealtime battles and/or
parental anxiety/stress
Usually demonstrates age
appropriate behaviors
Mealtimes reasonable length
(20–30 min)
Meals rarely consumed while
watching TV
Validation and reliability of NutriSTEPt
JA Randall Simpson et al
Moderate risk
BMI for age 85th percentile to o95th percentile
Recent illness, surgery or hospitalization
History of iron deficiency, treated with diet
Some problems with teeth or mouth that make it
difficult to eat or drink
Chronic GI problems which occur a few times a
week
Diet restricted in one food group
Minimal use of animal products including milk and
eggs
Occasionally eats at regular times throughout the
day (breakfast, lunch, supper)
Offered snacks few times a week
Eats some variety of foods
Drinks throughout the day and is not hungry at
mealtimes
Milk intake o2 or 43 cups a few days of the week
Juice intake 46 oz (3/4 cup) but p8 oz (1 cup)
daily
Eats fast foods 2–3 times a week
Sometimes given unsafe, inappropriate foods, for
example, raw eggs, unpasteurized milk, herbal
teas, pop, fruit drinks
Some food allergy, restriction or diet concerns
Some supplement concerns or issues
Frequent poor appetite with decreased intake
Some food restrictions or preferences or frequently
considered a ‘picky eater’ by parent
Frequent (daily) mealtime battles and/or parental
anxiety/stress
Occasionally demonstrates age appropriate
behaviors
Still being spoon fed by parent
Still drinking from a bottle
Usually spends a long time at meals (e.g. an hour)
or unable to sit for about 15 min at meals
Food is used as a reward or punishment frequently
Meals frequently consumed while watching TV
779
High risk
Height for age o5th percentile
BMI for age X95th percentile
BMI for age o5th percentile
Lengthy illness or medical condition
History of iron deficiency anemia, treated with diet
and medication
Significant problems with teeth or mouth that
make it difficult to eat or drink
Chronic GI problems which occur more than twice
a week
Diet restricted in more than one food group
Missing one food group entirely
Rarely eats at regular times throughout the day
(breakfast, lunch, supper)
Rarely offered snacks
Does not eat a variety of foods from the four food
groups
Milk intake o2 or 43 cups most days of the week
Juice intake 48 oz (1 cup) daily
Eats fast foods four or more times a week
Frequently given unsafe, inappropriate foods, for
example, raw eggs, unpasteurized milk, herbal
teas, pop, fruit drinks
Significant food allergy, restriction or diet concerns
Significant supplement concerns or issues
Chronic poor intake with decreased intake
Extreme food restrictions or preferences or always
considered a ‘picky eater’ by parent
Significant mealtime battles and/or parental
anxiety/stress with mealtimes rarely pleasant
Rarely demonstrates age appropriate behaviors
Meal length times always less than 15 min and/or
more than 1 h
Food is used as a reward or punishment most of
the time
Meals always/almost always consumed while
watching TV
European Journal of Clinical Nutrition
 Validation and reliability of NutriSTEPt
JA Randall Simpson et al
780
Table B1 Continued

Low risk Moderate risk High risk

Adult or parent role model and/or Meals seldom consumed with adult or parent role Meals rarely or never consumed with adult or
presence at mealtimes model and/or presence parent role model and/or presence
Some difficulty with eating, chewing, swallowing, Significant difficulties with eating, chewing,
gagging and/or oral tactile sensitivities swallowing, gagging and/or oral tactile sensitivities

Other risk factors and relevant issues or concerns

Adequate food storage and cooking
facilities
Sufficient income and food available
to offer quality, quantity and variety
at all times
Daily active play indoors and
outdoors
Time spent watching TV, using the
computer or playing video games is
limited to o3 h daily
Stable family, childcare, social,
emotional situation
Limited food storage and cooking facilities
Limited income and food available to offer quality,
quantity and variety at all times
Active play only a few times a week (less than once
a day)
Watches TV, uses computer or plays video games
for 3–4 h most days of the week
Changes or stresses in family, childcare or social life
influencing some changes in usual appetite and/or
intake and/or growth
Inadequate food storage and cooking facilities
Inadequate income and food available to offer
quality, quantity and variety at all times
Restricted or minimal daily active play
Watches TV, uses the computer or plays video
games for X4 h most days of the week
Changes or stresses in family, childcare or social life
influencing significant or prolonged changes in
usual appetite, intake and/or growth

References used for developing risk rating for NutriSTEP validation.

1 Dietitians of Canada, the College of Family Physicians of
Canada, Community Health Nurses Association of Canada,
Canadian Pediatric Society (2004). The use of growth charts
for assessing and monitoring growth in Canadian infants
and children. Can J Diet Prac Res 65, 22–32.
2 Ontario Society of Nutrition Professionals in Public
Health (2004). Pediatric nutrition guidelines for primary
health care providers. Ontario Society of Nutrition Profes-
sionals in Public Health. Available at http://www.osnpph.
on.ca/pdfs/pediatric_nutrition_guidelines.pdf.
3 York Region Health Department (2004). A quick
reference guide for early years professionals. Early
identification in York Region. Red flags. York Region
Health Department. Available at http://www.region.york.
on.ca/.
4 Health Canada (1995). Canada’s Food Guide to Healthy
Eating Focus on preschoolers. Background for educators and
communicators. Health Canada (H39-308/1995E). Available
at http://www.hc-sc.gc.ca/fn-an/food-guide-aliment/res/fg_
preschoolers-prescolaire_ga_e.html.
5 University of Montreal, Nestle Nutrition, and Dietitians
of Canada (2002). Healthy ABCs. Assessment from birth to
childhood (0–5 years). A problem-based learning program.
Resource guide for participants. Workshop materials from
Integrating Nutrition into Family and Paediatric Practice
Conference, Vancouver, BC.

European Journal of Clinical Nutrition