European Journal of Clinical Nutrition (2005) 59, 211–218

& 2005 Nature Publishing Group All rights reserved 0954-3007/05 $30.00
www.nature.com/ejcn

ORIGINAL COMMUNICATION
Validation of a food-frequency questionnaire
assessment of carotenoid and vitamin E intake using
weighed food records and plasma biomarkers:
The method of triads model
SA McNaughton1*, GC Marks1, P Gaffney2, G Williams3 and A Green4

1
Nutrition Program, School of Population Health, University of Queensland, Herston, Queensland, 4029, Australia; 2School of Health
Sciences, Griffith University, Gold Coast, Queensland, Australia; 3Tropical Health Program, School of Population Health, University of
Queensland, Herston, Queensland, 4029, Australia; and 4Queensland Institute of Medical Research, Herston, Queensland, 4029,
Australia

Background: Reliability or validity studies are important for the evaluation of measurement error in dietary assessment methods.
An approach to validation known as the method of triads uses triangulation techniques to calculate the validity coefficient of a
food-frequency questionnaire (FFQ).
Objective: To assess the validity of an FFQ estimates of carotenoid and vitamin E intake against serum biomarker measurements
and weighed food records (WFRs), by applying the method of triads.
Design: The study population was a sub-sample of adult participants in a randomised controlled trial of b-carotene and sunscreen
in the prevention of skin cancer. Dietary intake was assessed by a self-administered FFQ and a WFR. Nonfasting blood samples
were collected and plasma analysed for five carotenoids (a-carotene, b-carotene, b-cryptoxanthin, lutein, lycopene) and vitamin
E. Correlation coefficients were calculated between each of the dietary methods and the validity coefficient was calculated using
the method of triads. The 95% confidence intervals for the validity coefficients were estimated using bootstrap sampling.
Results: The validity coefficients of the FFQ were highest for a-carotene (0.85) and lycopene (0.62), followed by b-carotene
(0.55) and total carotenoids (0.55), while the lowest validity coefficient was for lutein (0.19). The method of triads could not be
used for b-cryptoxanthin and vitamin E, as one of the three underlying correlations was negative.
Conclusions: Results were similar to other studies of validity using biomarkers and the method of triads. For many dietary
factors, the upper limit of the validity coefficients was less than 0.5 and therefore only strong relationships between dietary
exposure and disease will be detected.
Sponsorship: National Health and Medical Research Council.
European Journal of Clinical Nutrition (2005) 59, 211–218. doi:10.1038/sj.ejcn.1602060
Published online 13 October 2004

Keywords: validity coefficient; dietary assessment; biomarkers; food-frequency questionnaire; weighed food record; method of
triads; carotenoids; vitamin E

*Correspondence: SA McNaughton, MRC Human Nutrition Research,

Elsie Widdowson Laboratory, Fulbourn Rd, Cambridge CB1 9NL, UK.
E-mail: sarah.mcnaughton@mrc-hnr.cam.ac.uk
Guarantor: SA McNaughton.
Contributors: AG, GW, GCM and PG were responsible for the design
and conduct of the study. SM conducted the data analysis, interpreta-
tion of the data and writing of the manuscript. All the authors
participated in editing the manuscript and provided advice regarding
interpretation of the results. None of the contributing authors had
any financial or personal interests in any of the bodies sponsoring this
research.
Received 31 July 2003; revised 22 April 2004; accepted 11 August 2004;
published online 13 October 2004
Introduction
Assessment of long-term dietary intake has generally been
associated with significant measurement errors. Reliability or
validity studies can be used to determine the validity
coefficient and quantify the degree of measurement error
in the dietary assessment method (Armstrong et al, 1992).
Most studies assess validity through the comparison of two
methods containing errors that are likely to be correlated,
resulting in distorted measures of validity (Armstrong et al,
1992).
 Validation of an FFQ using the method of triads
SA McNaughton et al
212
Kaaks (1997) describes an approach to measurement of
validity using biomarkers. They recommend that when
determining the validity coefficient of the questionnaire
measurement (rQT), at least two additional measurements of
dietary intake are necessary, for example, biomarker mea-
surements and weighed food records or 24-h recalls. The
triangular approach to validation known as the ‘method of
triads’ uses the correlations between each of the three
methods to calculate the validity coefficient. The following
equation is used to estimate the validity coefficient for the
questionnaire:
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
rQR
rQT ¼ rQB  ð1Þ
rBR
where: rQT is the validity coefficient for the questionnaire,
rQB the correlation between the FFQ and the biomarker, rRB
the correlation between weighed food record and biomarker
and rQR the correlation between FFQ and the weighed food
record.
This method has been described in further detail by Ocke
and Kaaks (1997). The major assumption of this method is
that the correlations between the three measurements are
explained by the fact that they are all linearly correlated with
the true intake and that their errors are independent. The
errors of biomarkers and of questionnaire measurements or
food records can be considered independent and uncorre-
lated, but since questionnaires and food records may have
common sources of error, the errors may be positively
correlated. Consequently, the correlation coefficient be-
tween the questionnaire and the food record will be over-
estimated. It is suggested that the value calculated by the
method of triads should be used as the upper limit of the true
validity coefficient, that is, rQT o rQT(triad) and a range for
the validity coefficient can be established by using the
correlation between the questionnaire measurement and the
biomarker measurement as an estimate of the lower limit of
the validity coefficient, that is rQB o rQT o rQT(triad) (Ocke
and Kaaks, 1997).
Although the method of triads was described a number of
years ago there have been few applications within dietary
validation studies. Many studies have used biomarkers for
validation of dietary intake; however, they have applied the
traditional model of validation using only two dietary
methods (for example, biomarkers vs an FFQ) and few
studies have interpreted their results with respect to the
validity coefficient. Currently, only three published studies
have applied the method of triads to the validation of dietary
data using serum/plasma biomarkers (Ocke and Kaaks, 1997,
Daures et al, 2000, Kabagambe et al, 2001).
The objective of this study was to determine the validity of
the estimates of carotenoid and vitamin E intake from the
FFQ used in the Nambour Skin Cancer Study using data from
serum biomarker measurements and WFRs applying the
method of triads. This study is among the first to use this
method to investigate carotenoids other than b-carotene and
vitamin E.
European Journal of Clinical Nutrition
Methods
Subjects
The main focus of the Nambour Skin Cancer Study was a
randomised controlled field trial of b-carotene and sunscreen
use for the prevention of keratinocytic skin cancer con-
ducted in the township of Nambour, 100 km north of
Brisbane (Green et al, 1994; 1999). In 1992, prior to the
beginning of the trial (baseline), a subset of the Nambour
Skin Cancer Study group was randomly selected to partici-
pate in a dietary intake validation study (Ashton et al, 1996).
A total of 168 subjects were invited into the study, with 115
subjects completing both the FFQ and the WFRs, a response
rate of 69%. A subset of these 115 subjects (selected
randomly) also had blood samples taken and analysed. A
sample of 30 subjects had blood samples analysed for
carotenoids, vitamin E and total cholesterol and therefore
had dietary exposure data available from all three methods.
Subjects were excluded from this analysis if they did not
respond to over 90% of items on the FFQ (n ¼ 1) or if
they completed less than 10 days of recording in the WFRs
(n ¼ 1).
Food-frequency questionnaire
Dietary intake was assessed by a self-administered, semi-
quantitative FFQ at baseline in 1992. The FFQ was based on a
questionnaire designed for use in the United States (Willett
et al, 1985) and adapted for use in Queensland by the
Nutrition Program, University of Queensland. The ques-
tionnaire consisted of 129 specific foods or food groups
(including 20 fruit items and 26 vegetable and legume items)
with nine response options ranging from ‘never’ to ‘4 þ
times per day’ for the frequency of consumption of specified
serving sizes. Questions on cooking methods, specific types
of fats, oils, margarines, breakfast cereals, takeaway foods
and self-prescribed nutritional supplements were also in-
cluded on the questionnaire. Subjects were asked to recall
their frequency of consumption over the preceding six
months.
Weighed food records
Subjects completed records on two nonconsecutive days,
every 2 months for 1 y during 1993. Starting days were
randomly allocated in the initial recording period and then
the rest of the week was worked through in subsequent
recording periods so that each day of the week was recorded
at least once. Detailed instructions on use of the scales and
the weighing and recording of food and drink consumed
including leftovers were provided for the participants during
the initial interview. Participants were asked to record
information on time and place of food and meal preparation,
brand names of products and recipes used. For meals
eaten away from home, participants were asked to provide
detailed descriptions and approximate amounts of what was

eaten. Details of any self-prescribed nutritional supplements
consumed (other than the trial supplement) were also
recorded.
Serum biomarkers
Nonfasting venous blood samples were collected at baseline
in 1992 using standard venipuncture techniques performed
by experienced phlebotomists. Blood samples were processed
at the time of collection and serum samples were stored in
approximately 1 ml aliquots at 701C until analysis.
Measurement of carotenoids, vitamin E and cholesterol
was conducted by Queensland Health Pathology Services at
the Royal Brisbane Hospital. Measurement of carotenoids
and vitamin E was conducted simultaneously using the
method of Sowell et al (1994). Serum was treated with
ethanol to precipitate proteins, and the carotenoids and
vitamin E were extracted using petroleum ether and analysed
by reverse-phase high-performance liquid chromatography
(HPLC) using acetonitrile:ethanol (60:40) with 0.1 ml/l
triethylamine on a Waters Novapak column. Values labelled
as lutein represent lutein and zeaxanthin as these were not
separated in the HPLC analysis. Total cholesterol was
measured using an enzymatic colorimetric test (Siedel et al,
1983; Katterman et al, 1984). This is based on the
determination of D4-cholestenone after enzymatic cleavage
of the cholesterol ester by cholesterol esterase, conversion of
the cholesterol by cholesterol oxidase and measurement by
the Trinder reaction of the hydrogen peroxide formed using
photometric methods.
Analysis
Dietary intake of carotenoids and vitamin E was calculated
using international food composition data (United States
Department of Agriculture, 1998; United States Department
of Agriculture, 1999) as these dietary factors were not
available in the standard Australian database (National Food
Authority, 1995). Nutrient intake from food and the
consumption of dietary supplements was analysed using
software and a supplements database (Ashton et al, 1997)
developed by the Nutrition Program, University of Queens-
land.
Dietary intake was calculated as intake from diet alone and
as intake from diet and supplements for those dietary factors
affected by supplement use. Total carotenoids were calcu-
lated as the sum of the five individual carotenoids for both
serum and dietary intakes. Energy intakes were investigated
but no extreme values as defined by Willett (1998) were
identified.
Intakes of dietary factors calculated from the FFQ and the
WFRs were adjusted for energy intake using the nutrient
residual method described by Willett (1998). It is recom-
mended that if dietary intakes are to be adjusted for energy
intake in studies of diet–disease relationships, measurements
of validity should also be adjusted for energy intake (Willett,
Validation of an FFQ using the method of triads
SA McNaughton et al
213
2001). A similar method of adjustment commonly used to
adjust serum carotenoids and vitamin E for cholesterol was
applied in the current study due to the underlying relation-
ship between serum carotenoids and vitamin E, and serum
cholesterol (Stryker et al, 1990; Mandel et al, 1997; Hunter,
1998).
Spearman correlation coefficients were calculated between
the three dietary assessment methods (ie FFQ and WFR, FFQ
and biomarker, WFR and biomarker) for each individual
dietary factor. The small number of subjects precluded the
analysis being undertaken separately for males and females
and across age groups or other potential confounding factors
(eg smoking status). Analyses were performed using SPSS for
Windows Version 10.0.5 (SPSS Inc., 1999).
The correlations between each of the three dietary
methods were used to calculate the validity coefficient via
the method of triads, using Equation (1). This is interpreted
as the upper limit of the validity coefficient whereas the
correlation between the biomarker and the FFQ is inter-
preted as the lower limit of the validity coefficient. The 95%
confidence intervals for the validity coefficients were
estimated using bootstrap sampling where 1000 samples of
equal size (n ¼ 28) were obtained with replacement from the
study subjects (Ocke & Kaaks, 1997; Kabagambe et al, 2001).
Results
Data for carotenoids and vitamin E for all three dietary
exposure measurements were available on 28 subjects.
Table 1 provides details on the general characteristics of
the study group including age, gender, body mass index
(BMI), smoking status and supplement use.
Table 2 presents the mean and standard deviation of the
serum biomarker measurements of each of the five carote-
noids, vitamin E and cholesterol. Table 3 presents the mean,
standard deviation and range of intakes of the dietary factors
from the WFRs and the FFQs. The mean FFQ intakes were
consistently higher than the mean intakes based on the
Table 1 Summary statistics for general subject characteristics
Variable
n 28
Gender Male 11 (39.3%)
Female 17 (60.7%)
Age (y) Mean 7 s.d. 48.0710.5
Range 27.0–69.0
BMI (kg/m2) Mean 7 s.d. 25.773.8
Range 19.6–34.7
Smoking status Nonsmoker 15 (53.6%)
Ex-smoker 10 (35.7%)
Current smoker 3 (10.7%)
Supplement usea Any supplement 11 (39.3%)
b-Carotene 2 (7.1%)
Vitamin E 9 (32.1%)
BMI, body mass index; s.d., standard deviation.
a
Based on data collected as part of the FFQ.
European Journal of Clinical Nutrition
 Validation of an FFQ using the method of triads
SA McNaughton et al
214
WFR. Carotenoid intakes based on the FFQ varied from while poor correlations were shown for b-cryptoxanthin,
1.6-fold higher for lycopene to 3.1-fold higher for lutein lutein and vitamin E (diet only). Correlations between the
compared to intake on the WFR. Vitamin E intakes based on WFR and the biomarkers were moderate for a-carotene,
the FFQ were 1.6-fold higher than on the WFR. b-carotene, b-cryptoxanthin, lutein, and total carotenoids
Intake of dietary factors from self-prescribed supplements while poor correlations were shown for lycopene and
was also assessed and dietary intakes were calculated based vitamin E. The correlations between the WFR and the FFQ
on intake from diet alone and intakes based on diet and were more consistent (r 4 0.29) except b-cryptoxanthin and
supplements. The number of subjects consuming supple- lycopene, which were less than 0.14.
ments based on the FFQ is shown in Table 1. Supplement The correlations between each of the three dietary
intake was also assessed on the WFR and eight subjects exposure measurements were used to calculate the validity
reported taking supplements containing vitamin E while no coefficient for the FFQ using the method of triads approach
subjects reported taking supplements containing b-carotene. and the results are shown in Table 4 along with the 95%
Table 3 includes the total intakes from diet and supplements confidence intervals. Table 4 also presents the range for the
for those dietary factors affected by supplement use. validity coefficient, where the upper limit is that calculated
The Spearman correlation coefficients between each of the by the method triads and the lower limit is the correlation
three dietary assessment methods are shown in Table 4. between the FFQ and the biomarker as described earlier.
Moderate correlations between the FFQ and the biomarker However, when using the method of triads, a validity
were shown for a-carotene, b-carotene (diet only), lycopene, coefficient cannot be calculated if one of the three correla-
total carotenoids and vitamin E (diet and supplements), tions is negative (Ocke & Kaaks, 1997). Therefore, for
b-cryptoxanthin and vitamin E, the method of triads could
not be used and only a lower limit for the validity coefficient
Table 2 Mean, standard deviations and range of the serum biomarkers is presented. The validity coefficients for a-carotene and
lycopene (0.85, 0.62) were the highest of all the dietary
Serum biomarker (mmol/l) Mean 7 s.d. Range
factors, followed by b-carotene (0.55) and total carotenoids
a-carotene 0.1170.08 0.01–0.32 (0.55) while the lowest validity coefficient was for lutein
b-carotene 0.6270.59 0.03–2.70 (0.19). The validity coefficients for the biomarker measure-
b-cryptoxanthin 0.2570.35 0.02–1.88
ments and the WFR have also been presented in Table 4 for
Luteina 0.3670.45 0.03–2.31
Lycopene 0.1770.12 0.02–0.61 completeness. Overall, the validity coefficients for the FFQ
Total carotenoids 1.5171.23 0.24–5.22 were higher than those of the serum biomarkers, however
Vitamin E 34.8711.4 20.0–55.0 the validity coefficients of the WFR tended to be higher than
Total cholesterol 5.3571.18 3.61–8.02
those of both the FFQ and biomarkers. The correlations
a
Values labelled as lutein represent lutein and zeaxanthin as these were not between the three methods and the validity coefficients were
separated in the HPLC analysis. also investigated when intakes were adjusted for body weight

Table 3 Mean, standard deviation and range of intakes of each dietary factor (diet only) as estimated by the WFRs and the FFQ

WFR FFQ

Dietary factor Mean 7 s.d. Range Mean 7 s.d. Range

Energy (kJ)a Diet 794672147 4765–12 591 988673179 4263–17 375

a-Carotene (mg) 16017856 122–3575 423472275 1134–9622

b-Carotene (mg) Diet 406772271 623–9249 1000274954 4457–21 188

Diet þ supplements N/Ab N/Ab 12198711 426 4457–64 911

b-cryptoxanthin (mg) 2137214 9–802 6267528 102–1946

Lutein (mg)c 5237264 151–1036 163171172 397–4281
Lycopene (mg) 2336v1464 66–7489 381372299 517–9752

Total carotenoids (mg) Diet 874172937 2775–13 956 2030678514 9089–36 005
Diet þ supplements N/Ab N/Ab 22502713 117 9089–73 725

Vitamin E (mg) Diet 7.8272.44 4.28–14.25 12.8075.12 4.82–23.06

Diet þ supplements 37.697119.84 4.28–635.69 51.44791.92 4.82–351.69

WFRs, weighed food records; FFQ, food-frequency questionnaire; s.d., standard deviation; N/A, not applicable.
a
1 kcal ¼ 4.184 kJ.
b
No change in intakes as no subject reported taking supplements containing b-carotene on the WFR.
c
Values labelled as lutein represent lutein and zeaxanthin as these were not separated in the HPLC analysis.

European Journal of Clinical Nutrition

 Validation of an FFQ using the method of triads
SA McNaughton et al
215
Table 4 Spearman correlation coefficients between each of the three dietary assessment methods and the validity coefficient as calculated by the
method of triads

Correlation coefficients Validity coefficientsa Range for the validity coefficientb

FFQ WFR Biomarker

Dietary factor rQB rRB rQR rQT 95% CI rRT 95% CI rBT 95% CI FFQ Biomarker WFR

a-carotene 0.36 0.19 0.38c 0.85 0.65 – 1.00 0.45 0.06 – 0.86 0.42 0.09 – 0.96 0.36 – 0.85 0.36 – 0.42 0.19 – 0.45
b-carotene 0.22 0.26 0.36 0.55 0.15 – 1.00 0.65 0.16 – 1.00 0.40 0.09 – 0.99 0.22 – 0.55 0.22 – 0.40 0.26 – 0.65
Diet only Diet þ supplements 0.12 0.26d 0.29 0.36 0.06 – 1.00 0.79 0.16 – 1.00 0.33 0.07 – 1.00 0.12 – 0.36 0.12 – 0.33 0.26 – 0.79
b-cryptoxanthine 0.002 0.42c 0.13 N/A N/A N/A 40.002 40.002 40.42
Luteinf 0.03 0.35 0.40c 0.19 0.05 – 0.71 1.00 0.58 – 1.00 0.16 0.09 – 0.98 0.03 – 0.19 0.03 – 0.16 0.35 – 1.00
Lycopene 0.19 0.07 0.14 0.62 0.12 – 1.00 0.23 0.16 – 1.00 0.30 0.08 – 1.00 0.19 – 0.62 0.19 – 0.30 0.07 – 0.23
Total carotenoids 0.28 0.33 0.35 0.55 0.10 – 1.00 0.64 0.13 – 1.00 0.51 0.12 – 1.00 0.28 – 0.55 0.28 – 0.51 0.33 – 0.64
Diet only Diet þ supplements 0.18 0.33d 0.31 0.41 0.10 – 1.00 0.76 0.12 – 1.00 0.44 0.11 – 1.00 0.18 – 0.41 0.18 – 0.44 0.33 – 0.76
Vitamin Ee 0.05 0.11 0.57c N/A N/A N/A 40.05 40.05 40.11
Diet only Diet þ supplements 0.31 0.02 0.34 N/A N/A N/A 4 0.31 40.31 40.02

FFQ, food-frequency questionnaire; WFR, weighed food record; rQB, correlation between the FFQ and the biomarker; rRB, correlation between weighed food record
and biomarker; rQR, correlation between weighed food record and the FFQ; rQT, validity coefficient of the questionnaire; rBT, validity coefficient of the biomarker;
rRT, ¼ validity coefficient for the WFR; N/A, not applicable.
a
All values 41.00 were truncated as this is the largest possible value.
b
The lower limit is rQB for the FFQ and the biomarker and rRB for the WFR, and the upper limit is calculated by the method of triads.
c
P o 0.05.
d
No change in correlation between diet only and diet and supplements as no subject reported taking supplements containing b-carotene on the weighed food
records.
e
An upper limit for validity coefficient using the method of triads cannot be calculated, as one of the individual correlations is negative.
f
Values labelled as lutein represent lutein and zeaxanthin as these were not separated in the HPLC analysis.

based on calculating intakes per kilogram of body weight. No
consistent or substantial differences were shown compared
to the energy-adjusted values (data not shown).
Discussion
The objective of this study was to determine the validity of
the FFQ used in the Nambour Skin Cancer Study using
dietary exposure data for five carotenoids and vitamin E
available from WFRs and serum biomarkers. The method of
triads was used to calculate validity coefficients for each of
the dietary factors and provides an upper limit for the
validity coefficient while the correlation between the
biomarker and the FFQ provides a lower limit for the validity
coefficient (Kaaks & Riboli, 1997).
The 95% confidence intervals for the validity coefficients
in this study are wide, with the upper limits for the intervals,
for all carotenoids except lutein, greater than one, which
reflect the small sample size. Heywood cases are those
validity coefficients which are greater than one, and occur
if the product of two of the correlations between the dietary
methods is greater than the third correlation (Ocke & Kaaks,
1997). Heywood cases may be due to either random
sampling variations in the correlations between dietary
methods or violations in the underlying model assumptions
(Ocke & Kaaks, 1997, Daures et al, 2000). The confidence
intervals in this study are similar to those shown in other
studies using the method of triads (Ocke & Kaaks, 1997;
Daures et al, 2000; Kabagambe et al, 2001). Most studies
investigating the validity of dietary exposures do not
investigate the confidence intervals of their estimates and
therefore provide no indication of precision.
The measurement of serum biomarkers in this study was
based on a single blood measurement. Van Kappel et al
(2001) investigated the reproducibility of serum levels of
carotenoids and a-tocopherol by taking three blood samples
at yearly intervals. They found that a single sample could
accurately rank individuals for a-carotene, b-carotene, lutein
and a-tocopherol.
It is important to note the different reference periods or
timeframes for each of the dietary exposure measures in this
study. The FFQ and blood measurement were made at
approximately the same time at the beginning of 1992.
The FFQ asked respondents to consider intake over the
previous 6 months and the serum biomarkers are likely to
represent the preceding weeks and months of dietary
exposure while the WFRs were completed approximately
1 y later. Despite differing time frames, relatively good
correlations have been shown between the WFRs and the
other two dietary exposure measures for some dietary factors
and the correlations are within the range found in other
dietary validation studies (Willett & Lenart, 1998). However,
the correlations shown between the WFRs and the biomar-
kers may be an underestimate of the true correlations due to
the differing time frames.
One of the limitations of this study is the small sample size
available for the analysis. Data for all three dietary assess-
ments were available on 28 subjects for the five carotenoids
and vitamin E. One of the implications of the small sample
size is that it did not allow investigation of the validity across

European Journal of Clinical Nutrition

 Validation of an FFQ using the method of triads
SA McNaughton et al
216
important subject characteristics such as age, sex and
smoking status. This is important as the validity of dietary
intake measures has been show to vary between males and
females (Nelson, 1997). Similarly, studies have shown that
the correlations between intake and plasma/serum biomar-
kers are generally lower among non-smokers and has been
shown for b-carotene, total carotenoids, b-cryptoxanthin
and lutein but not lycopene or vitamin E (Peng et al, 1995;
Margetts & Jackson, 1996; EPIC Group of Spain, 1997).
Carotenoids
The correlations between the biomarkers and the two dietary
assessment methods were similar to those seen in other
studies of similar size, with correlations for b-carotene
ranging from 0.16 to 0.50 (Le Marchand et al, 1994; Scott
et al, 1996; Mandel et al, 1997; Kanetsky et al, 1998).
Correlations for other carotenoids were 0.25–0.52 for a-
carotene, 0.10–0.64 for lutein, 0.04–0.15 for b-cryptox-
anthin and 0.47–0.55 for lycopene (Le Marchand et al, 1994;
Scott et al, 1996; Kanetsky et al, 1998).
Comparisons with other studies using the method of triads
is limited as only three published studies have applied the
method of triads approach using serum/plasma biomarkers,
however the results for b-carotene in our study are similar to
those of previous studies (Ocke & Kaaks, 1997; Daures et al,
2000; Kabagambe et al, 2001). The study by Kabagambe et al
(2001) is currently the only published study to investigate
carotenoids other than b-carotene and showed that
b-cryptoxanthin had the highest validity. However, no
validity coefficient could be calculated for b-cryptoxanthin
in the current study, as one of the underlying correlations
was negative.
An alternative approach for calculating the upper limit of
the validity coefficient for b-cryptoxanthin would be to use
the square root of correlation between WFR and FFQ as an
approximation of the upper limit as described in the model
for validation studies proposed by Armstrong (Armstrong
et al, 1992). This would provide a value of 0.36 as the upper
limit of the validity coefficient of the FFQ, and therefore the
range of the validity coefficient would be 0.002 to 0.36.
This provides an advantage over just using the correlation
between the FFQ and the biomarker as the lower limit,
especially considering the lack of a meaningful correlation
between the biomarker and the FFQ.
Serum carotenoid levels are not homeostatically con-
trolled and therefore are largely dependent on the content
of the diet (Olson, 1999). However, a range of physiologically
related factors may impact on the concentrations in serum
and the correlation between dietary intake and serum
carotenoids. These include the efficiency of absorption of
the carotenoids, metabolism of carotenoids, uptake by body
tissues, release from tissues back into plasma and the rates of
catabolism (Olson, 1999).
The absorption and bioavailability of carotenoids can vary
considerably depending on the food source or food matrix,
European Journal of Clinical Nutrition
the presence of promoters or inhibitors and processing
factors such as cooking. The carotenoids in orange fruits,
sweet potato and pumpkin are present in lipid droplets and
are more available than those in green leafy vegetables,
which are complexed with proteins, and those in carrots and
tomatoes, which are present as crystals (Castenmiller & West,
1998; Huang et al, 2000; Ribaya-Mercado, 2002). The amount
of dietary fat in the diet is known to impact on the
absorption of carotenoids as the formation of micelles in
the intestine requires the presence of dietary fat (Van Het
Hof et al, 2000b). The presence of protein or lecithin can also
improve carotenoid absorption through the enhancement of
micelle formation (Castenmiller & West, 1998) while fibre
has an inhibitory effect (Torronen et al, 1996). Processing of
foods affects carotenoid bioavailability through disruption of
cell walls (Van Het Hof et al, 2000b). Heating and
homogenisation of tomatoes has been shown to increase
the bioavailability of lycopene (Van Het Hof et al, 2000a)
while homogenisation of spinach increased the serum
response to b-carotene although not all carotenoids were
affected equally (Castenmiller et al, 1999).
Provitamin A carotenoids undergo metabolism to vitamin
A in the intestinal mucosa, irrespective of vitamin A status,
and are incorporated into chylomicrons for transport to the
liver, whereas non provitamin A carotenoids are absorbed
intact (Parker, 1997). The extent and efficiency of metabo-
lism, including the proportion of the ingested carotenoid
affected in this way, is not entirely clear. Therefore, serum
levels of provitamin A carotenoids may only partially reflect
their dietary intake (Crews et al, 2001).
The influence of all these factors means that while it is
possible for the estimated intakes of subjects to be equal, the
differing bioavailabilities due to the different sources of
carotenoids and the processing of foods and the coconsump-
tion of inhibitors and promoters of carotenoid absorption
may result in differing observed serum responses between
subjects. These differences in bioavailability may also
explain the variability in the strength of the correlations
between the different carotenoids shown in this study (Van
Het Hof et al, 1999).
Vitamin E
Serum vitamin E showed good correlations with the FFQ but
only when supplement intake was also considered. This is in
agreement with other studies that have investigated correla-
tions between serum vitamin E and other measures of dietary
intake with correlations ranging from 0.31 to 0.88 in studies
of similar size (Porrini et al, 1995; Dixon et al, 1996; Booth
et al, 1997; Mandel et al, 1997; Kanetsky et al, 1998).
However, correlations between the biomarker and WFR were
poor regardless of whether supplement intake was included.
The difference in correlations between the FFQ and the
biomarker, and the WFR and the biomarker suggests that the
FFQ provides a better estimate of intake than the WFR, or,

more likely it reflects the congruence between the reference
periods.
The method of triads could not be applied to vitamin E as
one of the individual correlations was negative and therefore
only a lower limit for the validity coefficient was established.
As described earlier for b-cryptoxanthin, the square root of
correlation between WFR and FFQ could be used as an upper
limit of validity coefficient for the FFQ, which would provide
an upper limit value of 0.75 for intake from diet alone and
0.58 for intake from diet and supplements.
There are a number of possible reasons for the poor
correlations between intake from diet alone and serum
vitamin E. Firstly, there were differences in the exposure
measures in that dietary intake in this study was calculated
as a-tocopherol equivalents, which are based on the amounts
and relative activities of the various tocopherols and
tocotrienols (United States Department of Agriculture,
1999); however, the biomarker measurement represents
serum a-tocopherol only. Similarly, for the subjects consum-
ing supplements, vitamin E from foods does not represent
their true exposure. However, subanalysis of the 15 subjects
who reported no use of supplements containing vitamin E
(on either the FFQ or WFR) showed equally poor correlations
(data not shown). An alternative explanation may relate to
the underlying nature of the diet–serum relationship.
Increased vitamin E intake does lead to increased serum
levels over time however studies have tended to use intakes
achievable only through use of supplements (Willett et al,
1983; Brown et al, 1994; Meydani et al, 1998). It is possible
that there is a poor relationship between intake and serum
levels at the range of intakes from diet alone and a good
relationship at the range of intakes achieved through dietary
supplements use. The strength of the relationship at levels of
intake shown by supplement-users may be sufficient to
overwhelm the lack of relationship in the range of intakes
shown by non-supplement-users. Finally, although the
inclusion of supplements results in an extension of the
range of intakes, this would not have caused the improve-
ment in correlations as Spearman correlation coefficients
were used in the analysis rather than Pearson correlation
coefficients which may be susceptible to this effect (Streiner
and Norman, 1991; Delcourt et al, 1994).
Conclusions
Many studies of dietary validation compare test and
reference measures that contain errors, which are correlated
resulting in flawed estimates of validity. It is recognised that
biomarkers provide advantages over other dietary assessment
with respect to their independent errors and while many
studies have used biomarkers as tools for validation, few
studies interpret their results in terms of the validity
coefficient and few studies have applied the method of
triads model. This study is among the first to assess the
validity of intake of carotenoids other than b-carotene and
vitamin E using the method of triads.
Validation of an FFQ using the method of triads
SA McNaughton et al
217
The upper limit of the validity coefficients for the FFQ was
less than 0.5 for b-carotene (diet and supplement), lutein,
total carotenoids (diet and supplements) and b-cryptox-
anthin (if the correlation between the FFQ and WFRs is used
as the upper limit). The upper limit for the validity
coefficient is above 0.5 for a- and b-carotene (diet only),
lycopene, total carotenoids (diet only), and vitamin E (if the
correlation between the FFQ and WFRs is used as the upper
limit). This suggests that for many dietary factors, only
strong relationships between exposure and disease will be
detected.
Acknowledgements
SA McNaughton was supported by a National Health and
Medical Research Council Public Health Postgraduate Re-
search Scholarship.
References
Armstrong BK, White E & Saracci R (1992): Principles of Exposure
Measurement in Epidemiology. New York: Oxford University Press.
Ashton B, Ambrosini GL, Marks GC, Harvey PW & Bain C (1997):
Development of a dietary supplement database. Aust. NZ. J. of
Public Health 21, 699–702.
Ashton BA, Marks GC, Battistutta D, Green AC & the Nambour Study
Group (1996): Under-reporting of energy intake in two methods
of dietary assessment in the Nambour Trial. Aust. J. Nutr. Diet. 53,
53–60.
Booth SL, Tucker KL, McKeown NM, Davidson KW, Dallal GE &
Sadowski JA (1997): Relationships between dietary intakes and
fasting plasma concentrations of fat-soluble vitamins in humans.
J. Nutr. 127, 587–592.
Brown CC, Kipnis V, Freedman LS, Hartman AM, Schatzkin A &
Wacholder S (1994): Energy adjustment methods for nutritional
epidemiology: the effects of categorization. Am. J. Epidemiol. 139,
323–338.
Castenmiller JJM & West CE (1998): Bioavailability and bioconver-
sion of carotenoids. Annu. Rev. Nutr. 18, 19–38.
Castenmiller JJM, West CE, Linssen JPH, Van Het Hof KH & Voragen
AGJ (1999): The food matrix of spinach is a limiting factor in
determining the bioavailability of b-carotene and to a lesser extent
in humans. J. Nutr. 129, 349–355.
Crews H, Alink G, Andserson R, Braesco V, Holst B, Maiani G,
Oversen L, Scotter M, Solfrizzo M, Van Der Berg R, Verhagen H &
Williamson G (2001): A critical assessment of some biomarker
approaches linked with dietary intake. Br. J. Nutr. 86, S5–S35.
Daures JP, Gerber M, Scali J, Asre C, Bonifacj C & Kaaks R (2000):
Validation of a food-frequency questionnaire using mulitple-day
records and biochemical markers: application of the triads
method. J. Epidemiol. Biostat. 5, 109–115.
Delcourt C, Cubeau J, Balkau B & Papoz L (1994): Limitations of the
correlation coefficient in the validation of diet assessment
methods. Epidemiology 5, 518–524.
Dixon ZR, Burri BJ & Neidlinger TR (1996): Nutrient density
estimates from an average of food frequency and food records
correlate well with serum concentration of vitamins E and the
carotenoids in free-living adults. Int. J. Food Sci. Nutr. 47, 477–484.
EPIC Group of Spain (1997): Relative validity and reproducibility of a
diet history questionnaire in Spain. III. Biochemical markers. Int. J.
Epidemiol. 26, S110–S117.
Green A, Battistutta D, Hart V, Leslie D, Marks G, Williams G,
Gaffney P, Parsons P, Hirst L, Frost C & the Nambour Prevention
Study Group (1994): The Nambour Skin Cancer and Actinic Eye
European Journal of Clinical Nutrition
 Validation of an FFQ using the method of triads
SA McNaughton et al
218
Disease Prevention Trial: design and baseline characteristics of
participants. Control. Clin. Trials 15, 512–522.
Green A, Wiiliams G, Neale R, Hart V, Leslie D, Parsons P, Marks GC,
Gaffney P, Battistuta D, Frost C, Lang C & Russell A (1999): Daily
sunscreen application and betacarotene supplementation in
prevention of basal-cell and squamous-cell carcinomas of the
skin: a randomised trial. Lancet 354, 723–729.
Huang CJ, Tang YL, Chen CY, Chen ML, Chu CH & Hseu CT (2000):
The bioavailability of b-carotene in stir or deep fried vegetables in
men determined by measuring the serum response to a single
ingestion. J. Nutr. 130, 534–540.
Hunter D (1998): Biochemical indicators of dietary intake. In
Nutritional Epidemiology ed., WC Willett, pp 174–243. New York:
Oxford University Press.
Kaaks R & Riboli E (1997): Validation and calibration of dietary
intake measurements in the EPIC project: methodological con-
siderations. Int. J. Epidemiol. 26, S15–S25.
Kaaks RJ (1997): Biochemical markers as additional measurements in
studies of the accuracy of dietary questionnaire measurements:
conceptual issues. Am. J. Clin. Nutr. 65, 1232S–1239S.
Kabagambe EK, Baylin A, Allan DA, Siles X, Spiegelman D & Campos
H (2001): Application of the method of triads to evaluate the
performance of food frequency questionnaires and biomarkers
as indicators of long-term dietary intake. Am. J. Epidemiol. 154,
1126–1135.
Kanetsky PA, Gammon MD, Mandelblatt J, Zhang ZF, Ramsey E,
Dnistrian A, Norkus EP & Wright Jr TC (1998): Dietary intake and
blood levels of lycopene: association with cervical dysplasia
among non-Hispanic, black women. Nutr. Cancer 31, 31–40.
Katterman R, Jaworek D, Moller G, G A, Bjorkhem I, Svensson L,
Borner K, Boerma G, Leijnse B, Desager JP, Harwengt C, Kupke I &
Trinder P (1984): Multi-centre study of a new enzymatic method
of cholesterol determination. J. Clin. Chem. Clin. Biochem. 22,
245–251.
Le Marchand L, Hankin JH, Carter FS, Essling C, Luffey D, Franke AA,
Wilkens LR, Cooney RV & Kolonel LN (1994): A pilot study on the
use of plasma carotenoids and ascorbic acid as markers of
compliance to a high fruit and vegetable dietary intervention.
Cancer Epidemiol. Biomarkers Prev. 3, 245–251.
Mandel CH, Mosca L, Maimon E, Sievers J, Tsai A & Rock CL (1997):
Dietary intake and plasma concentrations of vitamin E, vitamin C,
and beta carotene in patients with coronary artery disease. J. Am.
Diet. Assoc. 97, 655–657.
Margetts BM & Jackson AA (1996): The determinants of plasma beta-
carotene interaction between smoking and other lifestyle factors.
Eur. J. Clin. Nutr. 50, 236–238.
Meydani SN, Meydani M, Blumberg JB, Leka LS, Pedrosa M &
Diamond R (1998): Assessment of the safety of supplementation
with different amounts of vitamin E in healthy older adults. Am. J.
Clin. Nutr. 68, 311–318.
National Food Authority (1995) NUTTAB95. Nutrient Data Table for
use in Australia. Australian Government Publishing Service,
Canberra.
Nelson M (1997): The validation of dietary questionnaires. In Design
Concepts in Nutritional Epidemiology eds. BM Margetts and M
Nelson, pp 241–272. New York: Oxford University Press.
Ocke MC & Kaaks RJ (1997): Biochemical markers as additional
measurements in dietary validity studies: application of the
method of triads with examples from the European Prospective
Investigation into Cancer and Nutrition. Am. J. Clin. Nutr. 65,
1240S–1245S.
Olson JA (1999): Carotenoids. In Modern Nutrition in Health eds. ME
Shils, JA Olson, M Shike and AC Ross, pp 525–541. Philadelphia:
Lippincott Williams & Wilkins.
Parker RS (1997): Bioavailability of carotenoids. Eur. J. Clin. Nutr. 51,
S86–S90.
Peng YM, Peng YS, Lin Y, Moon T, Roe DJ & Ritenbaugh C (1995):
Concentrations and plasma-tissue-diet relationships of carotenoids,
retinoids, and tocopherols in humans. Nutr. Cancer 23, 233–246.
European Journal of Clinical Nutrition
Porrini M, Gentile MG & Fidanza F (1995): Biochemical validation of
a self-administered semi-quantitative food-frequency question-
naire. Br. J. Nutr. 74, 323–333.
Ribaya-Mercado JD (2002): Influence of dietary fat on b-carotene
absorption and bioconversion into vitamin A. Nutr. Rev. 60,
104–110.
Scott KJ, Thurnham DI, Hart DJ, Bingham SA & Day K (1996): The
correlation between the intake of lutein, lycopene and beta-
carotene from vegetables and fruits, and blood plasma concentra-
tions in a group of women aged 50-65 years in the UK. Br. J. Nutr.
75, 409–418.
Siedel J, Hagele EO, Ziegenhorn J & Wahlefeld AW (1983): Reagent
for the enzymatic determination of serum total cholesterol with
improved efficiency. Clin. Chem. 29, 1075–1080.
Sowell AL, Huff DL, Yeager PR, Caudill SP & Gunter EW (1994):
Retinol, a-tocopherol, lutein/zeaxanthin, b-cryptoxanthin, lyco-
pene, a-carotene, trans-b-carotene, and four retinyl esters in serum
determined simultaneously by reversed phase HPLC with multi-
wavelength detection. Clin. Chem. 40, 411–416.
SPSS Inc. (1999): Statistical Package for the Social Sciences (SPSS) for
Windows. Standard Version. Release 10.05. Chicago: SPSS Inc.
Streiner DL & Norman GR (1991): Health Measurement Scales: A
Practical Guide to their Development and Use. New York: Oxford
University Press.
Stryker WS, Stampfer MJ, Stein EA, Kaplan L, Louis TA, Sober A &
Willett WC (1990): Diet, plasma levels of beta-carotene and alpha-
tocopherol, and risk of malignant melanoma. Am. J. Epidemiol.
131, 597–611.
Torronen R, Lehmusaho M, Hakkinen S, Hanninen O & Mykkanen H
(1996): Serum b-carotene response to chronic supplementation
with raw carrots, carrot juice or purified b-carotene. In Natural
Antioxidants and Food Quality in Atherosclerosis and Cancer Preven-
tion eds. JT Kumpulainen and JT Salonen Cambridge, UK: Royal
Society of Chemistry.
United States Department of Agriculture (1998) USDA NCC Caro-
tenoid Database for U.S. Foods.
United States Department of Agriculture (1999) USDA Nutrient
Database for Standard Reference, Release 13.
Van Het Hof KH, De Boer BCJ, Tijburg LBM, Lucius IZ, West CE,
Hautvast JGAJ & Weststrate JA (2000a): Carotenoid bioavailability
in humans from tomatoes processed in different ways determined
from the carotenoid response in the trigylceride-rich lipoprotein
fraction of plasma after a single consumption and in plasma after
four days of consumption. J. Nutr. 130, 1189–1196.
Van Het Hof KH, Tijburg LBM, Pietrzik K & Weststrate JA (1999):
Influence of feeding different vegetables on plasma levels of
carotenoids, folate and vitamin C Effect of disruption of the
vegetable matrix. Br. J. Nutr. 82, 203–212.
Van Het Hof KH, West CE, Weststrate JA & Hautvast JGAJ (2000b):
Dietary factors that affect the bioavailability of carotenoids. J.
Nutr. 130, 503–506.
Van Kappel AL, Steghens JP, Zeleniuch-Jacqotte A, Chajes V, Toniolo
P & Riboli E (2001): Serum carotenoids as biomarkers of fruit and
vegetable consumption in the New York Women’s Health Study.
Public Health Nutr. 4, 829–835.
Willett W (ed.) (1998): Nutritional Epidemiology. New York: Oxford
University Press.
Willett W (2001): Dietary diaries versus food frequency question-
naires — a case of undigestible data. Int. J. Epidemiol. 30, 317–319.
Willett W & Lenart E (1998): Reproducibilty and validity of food-
frequency questionnaires. In Nutritional Epidemiology ed., W
Willett, pp 101–147. New York: Oxford University Press.
Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain C, Witschi J,
Hennekens CH & Speizer FE (1985): Reproducibility and validity of
a semiquantitative food frequency questionnaire. Am. J. Epidemiol.
122, 51–65.
Willett WC, Stampfer MJ, Underwood BA, Taylor JO & Hennekens
CH (1983): Vitamins A, E, and carotene effects of supplementation
on their plasma levels. Am. J. Clin. Nutr. 38, 559–566.