TECHNICAL PAPERS

ARTWORK HERE

Verifying Measurement Uncertainty

Using a Control Chart with
Dynamic Control Limits
John Song and Theodore Vorburger
Abstract: We propose a control chart with dynamic control limits that promotes the process of developing and refin-
ing an uncertainty budget and verifying the measurement uncertainty. The upper and lower dynamic control limits,
µ + 2σ and µ − 2σ, are calculated from an updated mean, µ, and standard deviation, σ, using the measurement history
for a check standard. The dynamic control limits are compared with the upper and lower fixed control limits, y + CL
and y − CL, where y represents the reference value of the check standard; and CL represent the fixed control limit which
is calculated from the specific, mostly random, components of the uncertainty budget of the calibration and the check
standard. Only the uncertainty components that affect the variation of repeated measurements of the check standard
are used to calculate the fixed control limits. If the dynamic control limits are close to the fixed control limits, it indi-
cates that the measurement reproducibility for the check standard supports the estimated specific components of the
uncertainty budget. Otherwise, these uncertainty components may be either over- or under-estimated, and they may
need to be revised.

1. Introduction process, and mistakes for either over- or under-estimated uncer-

Reporting the uncertainty of measurement is usually the last tainties may occur. In the Surface and Microform Metrology
step in a measurement procedure. It is often a time consuming Group of the Precision Engineering Division at the National
Institute of Standards and Technology (NIST), a standardized
John Song procedure for reporting uncertainty of measurements for NIST
surface roughness and step height calibrations was developed in
Theodore Vorburger
the early 1990’s, and revised in 2001 and 2006. [1] A set of
National Institute of Standards and Technology check standards, covering a range of step heights from 0.02937
Gaithersburg, MD 20899 USA
µm to 22.90 µm and roughness values, Ra, ranging from
Email: jun-feng.song@nist.gov
Ra = 0.3 µm to 3 µm, was repeatedly measured for measure-

38 | MEASURE www.ncsli.org

ment quality control. We have recently analyzed the historical
control charts using dynamic control limits. The analyzed
results support the uncertainty procedure and provide useful
information for future revisions.
The upper and lower dynamic control limits, µ ± 2σ, are cal-
culated from the updated mean, µ, and standard deviation, σ,
using the measurement history for a check standard. These
values are then compared with the upper and lower fixed
control limits, y ± CL, which are calculated from the reference
value, y, assigned to the check standard, and the control limits,
CL, which are calculated using only the special uncertainty com-
ponents of both the calibration and the check standard, which
affect the variation of repeated measurements the check stan-
dard. When the dynamic control limits are close to the fixed
control limits, it indicates that the measurement reproducibility
for the check standard supports the estimated specific uncer-
tainty components of the uncertainty budget. Otherwise, the
specific components of the uncertainty budget may be either
over- or under-estimated, and they may need to be revised.
In the following sections, we discuss the concepts of uncer-
tainty of measurement, working and check standards, control
charts and control limits, including both the fixed and the
dynamic control limits, and also discuss how to use the dynamic
control chart to verify the specific components of the uncer-
tainty of measurement.
2. Uncertainty of Measurement
According to the International Vocabulary of Basic and
General Terms in Metrology (VIM), the uncertainty of measure-
ment is a “parameter, associated with the result of a measure-
ment, which characterizes the dispersion of the values that
could reasonably be attributed to the measurand.” [2]
Uncertainty of measurement is calculated from an uncertainty
budget, which contains Type A and Type B uncertainty compo-
nents and their sensitivity coefficients [3], following a standard-
ized procedure to calculate the combined standard uncertainty,
uc, and the expanded uncertainty, U = 2 uc, or alternatively a 95
% confidence level. The measurement result is often reported as
Y = y ± 2 uc (k = 2) or Y = y ± U (1 − α = 95 %). [3]
The uncertainty budget is derived from statistical analyses
and experimental estimations for Type A and Type B uncertainty
components and their sensitivity coefficients, including, but not
limited to, those from the calibration standard, measurement
method, instrument, environment and operator. For the estima-
tion of an uncertainty of measurement for the same measure-
ment process, different people may have different models,
resulting in different uncertainty estimations in the uncertainty
budget. Even in an international key comparison conducted
among the national metrology institutes (NMIs), significant dif-
ferences may occur. [4] It is not always a question of which
result is “right” or “wrong.” However, if the estimated measure-
ment uncertainty is less than the band of the measurement
reproducibility, it is almost certain that the measurement uncer-
tainty is under-estimated. And as a matter of fact, an over- or
under-estimated measurement uncertainty may especially occur
during the developing stage of an uncertainty budget, resulting
in either a conservative or an aggressive uncertainty estimate.
Vol. 2 No. 3 • September 2007
TECHNICAL PAPERS
But how do we know if the measurement uncertainty in not
over- or under-estimated? How can metrologists promote the
process for developing and refining an uncertainty budget and
verifying the overall measurement uncertainty? One approach
is discussed in this paper.
3. Working Standard and Check Standard
Different physical standards, including primary standards, sec-
ondary standards, reference standards, working standards and
check standards as defined in the VIM [2], are used for differ-
ent purposes; for example: realizing the SI unit; transferring the
SI unit from a NMI to other laboratories for instrument calibra-
tion and measurement quality control; or establishing measure-
ment traceability to the SI unit. Both primary standards and
secondary standards are established and used at NMIs and local
laboratories. On the other hand, a working standard is a “stan-
dard that is used routinely to calibrate or check material meas-
ures, measuring instruments or reference material.” A check
standard is a “working standard used routinely to ensure that
measurements are being carried out correctly.” [2] Thus, in a
calibration laboratory, a working standard is mainly used for
instrument calibration, while a check standard is mainly used
for measurement quality control.
4. Control Chart and Control Limits
4.1 Fixed Control Limits
In order to ensure that measurements are being carried out cor-
rectly, a check standard, with its metrological characteristics
close to those of the measurand [5], is routinely measured. In
general, if the measured value is within the control limits, the
measurement is considered as valid. The control limits can be
calculated either from the special components of the uncertainty
budget of both the calibration and the check standard, or from
the updated measurement results of the measurement repro-
ducibility of the check standard. As a result, there are two dif-
ferent control limits in a control chart: the fixed control limits
and the dynamic control limits. In the following discussions, we
use a control chart for a NIST 0.02937 µm step-height check
standard to demonstrate how to calculate the fixed control limits
from the uncertainty budget and how to use the dynamic control
chart for verifying the special components of the measurement
uncertainty.
A dynamic control chart with both fixed and dynamic control
limits is shown in Fig. 1. This was developed for NIST step-
height calibrations using a stylus instrument. [1] We used a
0.09065 µm step (expanded uncertainty of 0.00044 µm, k = 2)
as a working standard for instrument calibration, and a 0.02937
µm step (expanded uncertainty of 0.00042 µm, k = 2) as a check
standard for measurement quality control. There are 28
control measurements of the check standard carried out from
1993 to 2004. The fixed upper and lower control limits, y ± CL,
are calculated from the equation:
y ± CL = y ± 2 u(CL) , (1)
where y is the reference value of the step height, 0.02937 µm,
MEASURE | 39
TECHNICAL PAPERS

0.02937 µm Dynamic Control Chart for 0.09065 µm Step

0.02937 µm Dynamic Control Chart for 0.09065 µm Step
0.0305

0.0300

Fixed upper CL
Step Height µm

0.0295 Dynamic upper CL

Measured values
Updated mean
0.0290
Dynamic lower CL
Fixed lower CL
0.0285
0.0305

0.0280
0.0300

99
95

8
3

01

03
7

98

4
98
3

99

99

99

99
99

99

00
9

19
19

20

20
19
19
19

Fixed upper CL
/1

/1

/1

/1
/1

/1

/2
8/
1/

6/

5/
1/
9/
5/

22

27

27

13
3

19

/1
/1
2/
5/

3/

3/
/3
/2
5/

10
Dynamic upper CL
2/

2/

9/

7/

0.0295
8/

11
12
12
Measured values
Measurement Date
Measurement Updated mean
0.0290
Dynamic lower CL
Fixed lower CL
0.0285

Figure 1. A dynamic control chart shows both fixed and dynamic control limits (CL). The 0.02937 µm step check standard was repeatedly
0.0280
measured by a stylus instrument calibrated by a 0.09065 µm working standard. The dashed line represents the reference value of 0.02937 µm.
99
95

8
3

01

03
7

98

4
98
93

99

99

99

99
99

99

00
19
19

20

20
19
19
19

/1

/1

/1

/1
/1

/1

/2
8/
1/

6/

5/
1/
9/
5/

22

27

27

13
3

19

/1
/1
2/
5/

3/

3/
/3
/2
5/

10
2/

2/

9/

7/
8/

11
12
12

determined by interferometric and stylus measurements, and
u(CL) is the combined standard uncertainty of the fixed control
limits which is calculated from the relevant specific, mostly
random, components of uncertainty budget for the measure-
ment process, which include components from both the
working standard (0.09065 µm step) and the check standard
(0.02937 µm step) which affect the variation of the check meas-
urement. For the report of uncertainty for NIST step height cal-
ibrations, four sources of standard uncertainty are included in
the uncertainty budget. In addition the four sources apply to
both the working standard and the check standard. The four
sources are [1]:
u1: Geometrical nonuniformity and surface finish of either the
step-height working standard used to calibrate the instru-
ment, or the check standard used for measurement quality
control. This leads to an uncertainty in the z-scale calibra-
tion constant for the instrument, and an uncertainty from
the check measurement using the check standard.
u2: Variations in the calibration constant(s) due to (a) noise in
the stylus instrument transducer, (b) surface topography in
the reference datum of the stylus instrument, (c) sampling
and digitizing processes in the controller, and (d) round-off
in the software computations.
u3: Variations in the step height values due to nonlinearity in
the instrument transducer.
u4: Standard uncertainty in the reference value of the step-
height working and check standard as determined from
interferometric and other measurements.
When calculating the fixed control limits, note that only the
special uncertainty components of the working (calibration) and
check standards (which affect the variation of repeat measure-
ments of the check standard), were taken into account. For
example, when the working standard (in this case the 0.09065
µm step) is used for instrument calibration, the calibration
uncertainty assigned to the standard will include all four uncer-
tainty components of the working standard u(w)1, u(w)2, u(w)3
and u(w)4. However, when the check standard (the 0.02037 µm
step) is used for the check measurements on the instrument cal-
ibrated by the working standard, only the first three uncertainty
components of the working standard, u(w)1 , u(w)2 and u(w)3 ,
are included, because only these affect the variation range of the
repeat measurements for the check standard. The last uncer-
tainty component for the working standard, u(w)4, or the stan-
dard uncertainty of the reference value of the working
(calibration) standard, has no effect on the variation range of
the repeat measurements for the check standard, and therefore,
should not be included in the calculation of the fixed control
limits for the check standard.
On the other hand, when the check standard (the 0.02937 µm
step) is repeatedly measured, only the first two uncertainty com-
ponents u(c)1 and u(c)2, or the uncertainty from the height uni-
formity and surface finish of the check standard and the
uncertainty from the random variations of the calibration con-
stant(s), affect the variation range of the check measurements.
Therefore, they need to also be included in the calculation of the
fixed control limits for the check standard.

40 | MEASURE www.ncsli.org

As a result, the combined standard uncertainty, u(CL), of the
fixed control limits of the check standard due to repeat measure-
ment variations can be calculated from the equation:
u(CL) = [u (w)1 2 + u (w)2 2 + (2)
2 2
u (w)3 + u (c)1 + u (c)2 ]2 1/2
, Otherwise, the uncertainty procedure may need to be revised.
where u(w)1, u(w)2 and u(w)3 represent the first three uncertainty
components for the working standard (the 0.09065 µm step);
and u(c)1 and u(c)2 represents the first two uncertainty compo-
nents for the check standard (the 0.02937 µm step).
4.2 Dynamic Control Limits
By using the updated mean and standard deviation, µ and σ, of
the repeat measurements for the check standard, the upper and
lower dynamic control limits, µ ± 2σ, are calculated, and are
shown in Fig. 1. In this case, it can be seen that the dynamic
control limits are not stable at the first few measurements, when
the sample size is very small. They become more stable after
about ten measurements.
The fixed and dynamic control limits can be used to verify the
estimated special components of uncertainty of measurement.
For a well established uncertainty budget, the influence quanti-
ties and their sensitivity coefficients should be carefully investi-
gated by experimental and theoretical ways; and the
measurement bias, if any, should be totally removed. Therefore,
the updated mean from the repeat measurements of the check
standard, µ, should be very close to the reference value of the
check standard, y. And the updated standard deviation, σ,
should be very close to the combined standard uncertainty of the
fixed control limits, u(CL), calculated from the special uncertainty
components of both the working and the check standards (see
Eq. 2), which affect the variation range of the repeat measure-
ments of the check standard. As a result, the dynamic control
limits, µ ± 2σ, must be very close to the fixed control limits,
y ± 2 u(CL), with a 95 % confidence level. For our example of
using the check standard, if the check standard is repeatedly
measured, we expect about 19 out of 20 measurement results to be
located within the upper and lower fixed control limits, y ± 2 u(CL).
Otherwise, if there is measurement bias, or if the uncertainty
components and their sensitivity coefficients are poorly estab-
lished, the measurement uncertainties might be either over- or
under-estimated, and the dynamic control limits will be signif-
icantly different from the fixed control limits.
5. Developing and Refining the Uncertainty Budget
and Verifying the Measurement Uncertainty
A dynamic control chart with both fixed and dynamic control
limits is a useful tool for developing and refining the uncertainty
budget and for verifying the measurement uncertainty. It can
also monitor the long term measurement variation for quality
control purposes. When a draft uncertainty procedure is first
developed, the Type A and Type B uncertainty components may
not be carefully investigated, and their sensitivity coefficients
may not be well established. As a result, the measurement
uncertainty might be either over- or under-estimated. By repeat-
edly measuring a check standard, a dynamic control chart with
Vol. 2 No. 3 • September 2007
TECHNICAL PAPERS
both fixed and dynamic control limits can be established. If the
upper and lower dynamic control limits µ ± 2σ are close to that
of the fixed control limits y ± 2u(CL), that indicates the measure-
ment reproducibility for the check standard supports the esti-
mated special components of the uncertainty procedure.
Establishing confidence in the measurement reproducibility
might take days, months or even years, if it is necessary to
monitor the long term variations for a measurement system
including the calibration and check standards, measurement
instruments, the measurement environment and the operator.
During that period, the uncertainty procedure may have been
continually revised.
From Fig.1, it can be seen that after about ten measurements,
the dynamic control limits, µ ± 2σ, are close to the fixed control
limits, y ± 2u(CL). This indicates that the check measurements
support the estimated special components of the uncertainty
procedure. The dynamic control limits also show a small trend
of downward drift after the year of 2001. It may be just a
random variation, because the results of check measurements
are still within the range of the fixed control limits. Or it may
show the beginning stage of a long-term downward drift, which
needs to be continually monitored, and possibly corrected.
Actually, after the initial draft of this paper, this check standard
has been remeasured three times. Measurement results have
confirmed that there is a downward drift for the step height
check standard. As a temporary step of correction, we have
expanded the uncertainty component u(c)1 to reflect the
change of the 0.02937 µm check standard. For the long run, we
plan to replace it with a new check standard.
In order to verify the estimated special components of the
uncertainty procedure for NIST step height calibrations [1], we
have analyzed nine control charts for step height check stan-
dards ranging from 0.02937 µm to 22.90 µm. These results
support the previously estimated components of the uncertainty
procedures and provide useful information for future revision.
6. Summary and Discussion
A control chart composing fixed and dynamic control limits is pro-
posed for promoting the process of developing and refining the
uncertainty budget and verifying the measurement uncertainty.
Historical results of the control measurements for NIST step
height check standards covering a range from 0.02937 µm to
22.90 µm are analyzed using the proposed dynamic control chart.
The analyzing results support the estimated special components of
the uncertainty procedure for NIST step height calibrations. [1]
Measurement bias [2, 6] has a significant effect on the meas-
urement uncertainty. It is required that any measurement bias
must be removed from a measurement result. [3] However,
uncorrected bias [6] and unknown bias may exist in a measure-
ment result. Significant measurement bias could be found not
only in the measurements of local laboratories and industry, but
also in the international key comparisons conducted among
national metrology institutes. [4]
When the uncorrected bias is included in the uncertainty
budget, rather than used to correct the measured value, it
increases the measurement uncertainty. [6] Although the pro-
MEASURE | 41
TECHNICAL PAPERS
posed dynamic control chart can be used for verifying the esti-
mated special, mostly random, components of the uncertainty
procedure, it can’t guarantee a measurement result without any
uncorrected or unknown bias to the reference value. In other
words, the dynamic control chart is mainly used for a “one-way”
check, or for the check of the under-estimated measurement
uncertainties. If the band of the dynamic control limits, µ ± 2σ,
are significantly larger than that of the fixed control limits,
y ± 2u(CL), it is almost certain that the measurement uncertainty
is under-estimated. On the other hand, if the band of the
dynamic control limits is slightly less than that of the fixed
control limits, it is still unknown whether or not the uncertainty
is really over-estimated, because some uncorrected bias and
other factors might have been considered in a conservative esti-
mate for both the uncertainty budget and the fixed control limits.
This increases the band of the fixed control limits, but they
cannot be tested by repeat measurements of the check standard.
In order to verify any possible measurement bias, it is neces-
sary to conduct independent measurements by different instru-
ments with different calibration standards, or conduct
inter-laboratory comparisons. A “good” measurement result
would be demonstrated not only by its high accuracy and low
uncertainty, but also by the capability of reproducing the results
in close agreement at different times, even by different operators
using different calibration standards and instruments.
7. Acknowledgements
The authors are grateful to Mr. T. B. Renegar and Mr. C. D.
Foreman at the NIST Surface Calibration Laboratory for their
42 | MEASURE
contributions in the development of the uncertainty procedure
and control charts for NIST surface roughness and step height
calibrations.
8. References
[1] See Appendix A, “Measurement Conditions and Sources of Uncer-
tainties for NIST Surface Roughness and Step Height Calibration
Reports,” last revised in June 2006, at the web site
www.mel.nist.gov/div821/webdocs-13/surfcalib.htm.
[2] BIPM, IEC, IFCC, ISO, IUPAC, IUPAP and OIML, International
Vocabulary of Basic and General Terms in Metrology, ISO,
Geneva, 1993.
[3] BIPM, IEC, IFCC, ISO, IUPAC, IUPAP and OIML, Guide to the
Expression of Uncertainty of Measurement, ISO, Geneva, 1993.
[4] L. Koenders, R. Bergmans, J. Garnaes, J. Haycocks, N. Korolev, T.
Kurosawa, F. Meli, B.C. Park, G.S. Peng, G.B. Picotto, E. Prieto,
S. Gao, B. Smereczynska, T. Vorburger, and G. Wilkening, “Com-
parison on Nanometrology: Nano 2 – Step height,” Metrologia,
vol. 40, Technical Supplement, 04001, 2003. Available from
www.iop.org/EJ/abstract/0026-1394/40/1A/04001 and kcdb.bipm.org.
[5] J.F. Song and T.V. Vorburger, “Standard Reference Specimens in
Quality Control of Engineering Surfaces,” J. Res. Natl. Inst. Stand.
Technol., vol. 96, pp. 271–289, 1991.
[6] S.D. Phillips, K.R. Eberhardt, and B. Parry, “Guidelines for
Expressing the Uncertainty of Measurement Results Containing
Uncorrected Bias,” J. Res. Natl. Inst. Stand. Technol., vol. 102, pp.
577–585, 1997.
www.ncsli.org