Professional Documents
Culture Documents
Statistically Accurate Estimation of Hormone Concentrations and Associated Uncertainties: Methodology, Validation, and Applications
Statistically Accurate Estimation of Hormone Concentrations and Associated Uncertainties: Methodology, Validation, and Applications
Statistically Accurate Estimation of Hormone Concentrations and Associated Uncertainties: Methodology, Validation, and Applications
Automation and
Analytical Techniques
We describe a data reduction procedure to assign statistically accurate estimates of unknown hormone concentrations, with associated uncertainties, based on experimental uncertainties in sample replicates and the fitted
calibration curve. Three mathematical calibration curve
functions are considered. The one providing optimal
statistical characterization of reference calibrators is
chosen for unknown evaluation. Experimental error is
addressed by assigning and propagating uncertainty
estimates for each measured response (including zerodose responses) by an empirically determined discrete
uncertainty profile and by propagating calibration curve
uncertainty. Discrete uncertainty profiles account for
both response precision (replicability) and accuracy (deviation from predicted calibration curves) without relying on assumed theoretical response varianceassay
response relations. The validity of assigning variable
response weighting by this procedure was assessed by
Monte Carlo simulations based on chemiluminescence
growth hormone calibration curves. Much-improved
accuracy and estimated precision are achieved for unknown hormone concentrations, particularly extremely
low concentrations, by using this variable response
weighting procedure.
Since the development of RIAs in the early 1970s and their
widespread application to clinical chemistry, enhanced
sensitivity and precision have been achieved by nonradioactive indicator techniques such as fluorometry and
chemiluminescence. Empiric fitting of the calibration
curves so generated has not always kept pace with the
enhanced physical precision of these methods. For exam-
Department of Internal Medicine, Division of Endocrinology and Metabolism, National Science Foundation Center for Biological Timing, Gilmer Hall,
University of Virginia, Charlottesville, VA 22903.
1
Department of Pharmacology, University of Virginia Health Sciences
Center, Charlottesville, VA 22908.
*Author for correspondence. Fax 804-982-4505; e-mail ms3g@virginia.edu.
Received May 1, 1997; revision accepted September 5, 1997.
116
117
sive empirical validation of our concept of error propagation via Monte Carlo simulations, which examine the
properties and behavior of this data reduction procedure.
The basis for the Monte Carlo simulation experiments is a
set of highly consistent assay data from 14 growth hormone (GH) chemiluminescence assays [7], wherein high
sensitivity and well-defined experimental uncertainty estimates were desirable.2
Methods
To accommodate the largely monotonic increases or decreases in immunologically based calibration curves, we
initially evaluated three empiric algebraic functions, with
the forms given below. These functions include modifications of the widely used four-parameter logistic function
[1, 8, 9] and a modified four-state Adair expression.
MOD4P 2 ^R i& 5
Y` 2 Y0
^R i& 5 Y 0 1
N
D O
j51
11
j~@H# i/j
N
j51
A2D
1 1 ~@H# i/10
logC 10 logB
1D
2
Nonstandard abbreviations: GH, growth hormone; SSR, sum of squared
residual; and LH, lutropin.
logB
1D
10 O j)j
A2D
All calibration curve parameter estimations are performed by a modified GaussNewton nonlinear leastsquares parameter estimation algorithm [10, 11] to a convergence criterion of ,1026 relative change in variance
of fit.
MONOTONE 2
N
SDi 5
O ~R 2 ^R &!
ni
ij
j51
ni
1/ 2
where ni is the number of replicate responses at concentration i, and Rij is the jth response at concentration i.
Distance-proportional nearest-neighbor smoothing is then
used to generate smoothed estimates for a discrete uncertainty profile, ,SDi., by smoothing interior points as
^SDi& 5
and endpoints as
^SD1 & 5
^SDn& 5
118
WSSR~ a !
n2p
~H T H! jk 5
i51
22
O
^@H#& 5
O
n
i51
~@H# i/ s i2 !
n
i51
~1/ s i2 !
s mean 5
1
n
i51
~1/ s i2 !
Chemiluminescence
response (relative
light units)
Target
response SD
0
0.0025
0.01
0.04
0.12
0.4
1.33
4.6
13.3
45
620
730
1080
2390
5800
16 500
44 000
98 000
146 000
180 000
50
50
90
270
700
2000
5000
10 000
15 000
19 000
119
Results
calibration curve analysis of a synthetic gh
calibration curve
Simulations were based on 14 well-characterized GH
chemiluminescence assays [7]. Fig. 1 illustrates our comprehensive calibration curve analysis with the discrete
response uncertainty profile applied to five uniformly
distributed replicates.
Fig. 1. Calibration curve with corresponding discrete response uncertainty profile (inset).
Synthetic calibration curve input data for this example were of uniformly spaced
quintuple replicates representative of the conditions replicated during Monte
Carlo simulations as taken from selected data reported in Chapman et al. [7].
The best fit (lowest absolute SSRs) was obtained with 4PARMS.
Uniform
Variable
No. of
replicates
MONOTONE
4PARMS
MOD4P
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
77.6
81.9
86.3
87.5
89.5
79.4
82.5
85.8
87.8
89.4
69.3
62.1
61.9
63.3
62.3
5.5
2.2
2.3
1.4
1.7
4.8
2.1
2.2
1.3
1.9
11.7
9.8
10.8
11.3
11.8
16.9
15.9
11.4
11.1
8.8
15.8
15.4
12.0
10.9
8.7
19.0
28.1
27.3
25.4
25.9
120
Figure 4 shows results of high-replicate-number calibration curve analysis of a GH chemiluminescence assay (GH
Chemi) and a lutropin (LH) IRMA. For each set of
calibration curve data, analysis was performed by using
each of the three calibration curve functions in which
assay responses were weighted variably, uniformly, and
uniformly excluding zeroes. For both the GH Chemi and
the LH IRMA, the MONOTONE calibration curve function provided the lowest absolute SSRs for each of the
three assay response weighting schemes. Plotted in Fig. 4
are the three resulting MONOTONE calibration curves
for each weighting scheme. The three curves are nearly
superimposable in each case, but do deviate somewhat at
the lowest (zero) hormone concentration, particularly for
the GH Chemi. The error bars on the points and the plots
of estimated assay response error (insets) are those for the
121
Discussion
As part of a systematic characterization of experimental
uncertainty inherent in assay measurements, we have examined the performance of three monotonically varying algebraic forms for the assay doseresponse function, and delineated the joint experimental uncertainties inherent in the
fitted curve, assay replicates, calibrator replicates, and zerodose tubes. Among the three common fitting functions
explored (modifications of the logistic and Adair expressions), the modified four-state Adair equation (MONOTONE) was favored by the majority of calibration curve
realizations evaluated here, whether composed of one, two,
three, four, or five replicates per dose. A modified fourparameter logistic function [4PARMS], however, very commonly used for calibration curve analysis [1, 8, 9], was also
adaptable. We further demonstrated that the variable response weighting protocol exhibited performance superior
to that of either the uniform response weighting protocol or
the protocol involving uniform response weighting without
consideration of zero-hormone-concentration calibrators,
since variable weighting provided greater accuracy and
precision, especially in determining low-end hormone concentrations. Indeed, single-replicate conditions with variable
weighting provided better performance than even the quintuple-replicate cases involving uniform weighting with or
without utilization of zero calibrators at concentrations below ;0.04 mg/L in the Monte Carlo-simulated GH chemiluminescence assay. Approximately equivalent performance
among the three protocols was seen at higher hormone
concentrations.
We further observed that all of the protocols were inaccurate and imprecise at 45 mg/L in the Monte Carlosimulated GH chemiluminescence assay, illustrating the
importance of characterizing critically the relevant operating
range of any given assay configuration. With few replicates,
122
Fig. 4. High-replicate-number calibration curve analysis of a GH chemiluminescence assay (GH Chemi) and a LH IRMA showing plotted
results of MONOTONE best-fit calibration curves for variable, uniform,
and uniform without zeroes assay response weighting (nearly superimposable curves except near the zero-dose concentration).
Error bars and estimated assay response error inset plots are for variably
weighted assay response analysis.
agating the effects of calibration curve uncertainty contribute noticeable effects on concentration uncertainty estimates
beyond that due solely to response variability. Evidence of
this is provided by our observation that GH concentration
SDs were conservatively estimated by the variable weighting procedure when compared with directly estimated
Monte Carlo [GH] SDs. Monte Carlo estimates were generally lower than those provided by the variable weighting
protocol because calibration curve parameter uncertainty is
not propagated as a contributor in the direct Monte Carlo
estimates. That the Monte Carlo procedure is valid is supported, however, by the observation that Monte Carlo estimates of assay response SDs were extremely consistent (a
situation in which agreement should indeed be expected).
To our knowledge, uncertainty in the fitted parameters of
the calibration curve is not reflected in sample uncertainty
estimates in most available data reduction methods. Hence,
earlier procedures for calculating within-sample SDs underestimate sample variance. This bias is especially significant
in defining assay sensitivity, leading potentially to inferred
greater sensitivity than actually achievable. In addition,
underestimation of (low-end) assay uncertainty may have
nontrivial impact on computer-assisted curve fitting of
(weighted) neurohormone time series, presumptively promoting false-positive (type I) statistical errors. Lastly, estimating the precision of inferred statistics from a time series,
e.g., the SD of an approximate entropy estimate for any
given time series, will lead to an overstatement of precision.
In summary, the variable weighting data reduction protocol
described here provides greater accuracy and precision than
most commonly used hormone concentration data reduction procedures, particularly at extremely low hormone
concentrations. Three monotonically sigmoidal functional
forms for evaluating calibration curves are examined, after
which selection of a preferred model is based on empirical
grounds (lowest absolute SSRs). Assay responses are variably weighted by an empirically derived discrete assay
response uncertainty profile that (a) is specifically tailored to
the particular calibration curve response profile being considered yet free of any constraints applied by assuming a
particular functional form for a variance profile [1 6], (b)
accounts for both response precision (replicability) and accuracy (relative deviation from predicted calibration curve),
and (c) is generated in a manner maximally consistent with
the most probable derived calibration curve. We show that,
in principle, uncertainty estimates for both assay responses
and hormone concentrations can be obtained from even
single-replicate assay protocols. However, the reliability of
measures rises significantly upon increasing to duplicates.
Uncertainty in determination of the calibration curve is also
evaluated and subsequently propagated as a contribution to
derived concentration uncertainty estimates. The explicit use
of zero-hormone reference information during evaluation of
calibration curves also contributes to better determination of
low hormone concentrations. Efforts are currently under
way to fully implement this data reduction protocol into a
123
0
1
3
6
10
12
15
20
25
30
35
40
No. of replicates
8
8
8
8
8
8
8
8
8
8
8
8
GHavg
0.000
0.997
3.08
5.57
10.41
11.89
15.14
20.25
22.22
30.6
33.4
40.1
Arith.
0
1.5
2.5
5
7.5
12.5
20
40
80
120
160
LHavg
9
10
11
10
11
11
10
11
11
11
10
20.14
1.32
3.22
5.09
7.14
12.59
19.67
40.6
79.0
120.2
157.6
Arith.
Arith.
SEM
GHavg
SEM
GHavg
SEM
GHavg
SEM
0.029
0.040
0.10
0.18
0.29
0.29
0.28
0.56
0.83
1.5
1.7
2.5
20.0003
1.005
3.087
5.580
10.42
11.88
15.17
20.274
24.04
31.29
34.0
41.3
0.0045
0.027
0.028
0.055
0.14
0.19
0.15
0.092
0.96
0.98
1.1
1.0
20.0933
1.063
3.207
5.628
10.34
11.80
15.11
20.340
24.18
31.41
34.0
40.70
0.0056
0.030
0.028
0.053
0.13
0.19
0.15
0.094
0.98
0.97
1.0
0.93
20.3678
0.935
3.206
5.661
10.35
11.78
15.07
20.274
24.13
31.40
34.0
40.72
0.0061
0.033
0.029
0.054
0.13
0.18
0.15
0.094
0.98
0.98
1.0
0.93
Var-wgt.
[LH], IU/L
Uniform
Arith.
SEM
LHavg
0.32
0.35
0.39
0.42
0.38
0.44
0.66
1.2
2.1
3.4
6.0
20.15
1.47
3.25
5.04
7.13
12.67
19.83
40.92
79.3
122.4
159.8
We acknowledge support from: NSF DIR8920162 (National Science Foundation Center for Biological Timing; M.S.,
M.L.J., J.D.V.); NIH RR00847 (General Clinical Research
Center at the University of Virginia; M.L.J., J.D.V.); NIH
DK38942 (Diabetes and Endocrine Research Center at the
University of Virginia; M.L.J., J.D.V.); NIH RR08119 (Center for Fluorescence Spectroscopy at the University of
Maryland at Baltimore; M.L.J.); NIH GM35154 (M.L.J.);
NIH RCDA1K04 HD00634 (J.D.V.); NIH P30 HD28934
(Reproduction Research Center at the University of Virginia; J.D.V.); Baxter Healthcare Corp., Round Lake, IL
(J.D.V.); The NIH-supported Clinfo Data Reduction Systems; The Pratt Foundation; and The University of Virginia Academic Enhancement Fund.
References
1. Rodbard D, Lenox RH, Wray HL, Ramseth D. Statistical characterization of the random errors in the radioimmunoassay dose
response variable. Clin Chem 1976;22:350 8.
2. Rodbard D. Statistical estimation of the minimal detectable concentration (sensitivity) for radioligand assays. Anal Biochem 1978;90:112.
Arith.
SEM
0.17
0.28
0.25
0.39
0.19
0.29
0.48
0.83
1.1
3.1
2.6
LHavg
20.12
1.43
3.16
4.90
6.95
12.47
19.71
41.04
78.9
121.4
159.7
Arith.
SEM
0.16
0.27
0.25
0.39
0.19
0.29
0.48
0.83
1.1
3.1
2.7
LHavg
20.21
1.37
3.11
4.88
6.95
12.50
19.75
41.03
78.8
121.5
159.6
SEM
0.16
0.27
0.25
0.39
0.19
0.29
0.48
0.83
1.1
3.1
2.7
3. Sadler WA, Smith MH. Estimation of the response-error relationship in immunoassays. Clin Chem 1985;31:18025.
4. Davidian M, Carroll RJ, Smith W. Variance functions and the minimum
detectable concentration in assays. Biometrika 1988;75:54956.
5. Davidian M. Estimation of variance functions in assays with possibly
unequal replication and nonnormal data. Biometrika 1990;77:4354.
6. Hwang L-J. Impact of variance function estimation in regression
and calibration. Methods Enzymol 1994;240:150 70.
7. Chapman IM, Hartman ML, Straume M, Johnson ML, Veldhuis JD,
Thorner MO. Enhanced sensitivity growth hormone (GH) chemiluminescence assay reveals lower postglucose nadir GH concentrations
in men than women. J Clin Endocrinol Metab 1994;78:13129.
8. Rodbard D. Statistical quality control and routine data processing
for radioimmunoassays and immunoradiometric assays. Clin
Chem 1974;20:125570.
9. Rodbard D, Munson PJ, DeLean A. Improved curve-fitting, parallelism testing, characterization of sensitivity and specificity, validation, and optimization for radioligand assays. In: Radioimmunoassay and related procedures in medicine. Vienna: International
Atomic Energy Agency, 1977:1:469 509.
10. Johnson ML, Frasier SG. Nonlinear least squares analysis. Methods Enzymol 1985;117:301 42.
11. Straume M, Frasier-Cadoret SG, Johnson ML. Least-squares analysis
of fluorescence data. In: Lakowicz JR, ed. Topics in fluorescence
spectroscopy, Vol. 2: Principles. New York: Plenum, 1991:177241.
12. Bevington PR. Data reduction and error analysis for the physical
sciences. New York: McGraw-Hill, 1969:73.