Useful Tips for Presenting Data and

Measurement Uncertainty Analysis

Ben Smarslok

University of Florida 1
Mechanical and Aerospace Engineering
 Outline
• Why is presenting data properly important?

• Explain important terminology and definitions

– NIST vs. ISO vs. ASME/ASTM
– Oberkampf definitions of model uncertainty (not included)

• Experimental scenarios and corresponding methods

– Uncertainty propagation
– Crossed vs. nested factors (ANOVA vs. VCA)
– p-values
– Interlaboratory Studies (not included)

University of Florida 2
Mechanical and Aerospace Engineering
 m = 75 ± 5 g
What is the meaning of ± 5 ?
• Best guess by experimenter
• Half the smallest division of measurement
• Standard deviation: 
• Standard error: m = /n
• Expanded uncertainty of ± 2 or ± 3 (95% or 99%
confidence interval)
• Standard uncertainty: u
• Combined standard uncertainty: uc

*(Courtesy of Duane Deardorff presentation from UNC)

University of Florida 3
Mechanical and Aerospace Engineering
 What does x ± u mean?
• Engineers think in terms of ±2 (95%)
• Physicists generally report ±1 (68% CI)
• Chemists report ±2 or ±3 (95% or 99% CI)
• Survey/poll margin of error is 95% CI
• Accuracy tolerances are often 95% or 99%
• NIST Calibration certificate is usually 99%

• Conclusion: The interpretation of ± u is not consistent

within a field, let alone between fields
– It is very important to explain the statistical relevance of the
uncertainty bounds!!!

University of Florida 4
Mechanical and Aerospace Engineering
 Presenting Uncertainty Precisely
• Choose a standard for presenting uncertainty (I prefer
NIST), and reference the standard
• Explain the source of the uncertainty
– Type A – calculated by statistical methods (it is useful to explain
the design of experiments and the number of samples involved)
– Type B – determined by other means, such as estimate from
experience or manufacturers specifications
• Use terms carefully!
– Error vs. Uncertainty: Error is the deviation from the true value and
measured value (never known), which is estimated as uncertainty
– Bias vs. variability (will explain later)
• Avoid use of ambiguous ± notation without explanation
sx
• Pet peeve: CV  coefficient of variation 
– COV = covariance x

University of Florida 5
Mechanical and Aerospace Engineering
 NIST Classification of Measurement Uncertainties
• Uncertainty classification:
– Random uncertainty /
variability – scatter in the
measurements (v)
– Systematic uncertainty / bias –
systematic departure from the
true value (b)
• Type of evaluation:
– Type A – calculated by sx
v  s  xt = true value of specimen
statistical methods x x N  = experimental population average
– Type B – determined by other
x = experimental sample average
means, such as estimate from
experience vx = random error of sample
B Range is at 95% (2) level x = systematic error of sample
b
2 of a normal distribution

University of Florida 6
Mechanical and Aerospace Engineering
 Uncertainty Analysis Example
• Consider our transverse modulus work (E2)

• Hooke’s Law:
P = Load
P 2

E2  A = Area A  w  t
A 2
1

= transverse strain

• We will work through this problem backwards

Total Bias &

Components Contributors
Uncertainty Variability

University of Florida 7
Mechanical and Aerospace Engineering
 Level 1: Total Uncertainty
• In general,
U X 95   t95 (v X )2  (bX ) 2
– where, vX and bX were propagated from component uncertainties
t95 = Student’s t distribution at 95% confidence level
(depends on # of DOF)

• Total uncertainty of E2 at 1 (68%) confidence for

comparison to experimental results
U 68 E 2   (v E 2 ) 2  (bE 2 ) 2

• Or, at the commonly accepted 95% level

E2,95%  9.01  0.12GPa

University of Florida 8
Mechanical and Aerospace Engineering
 Level 2: Uncertainty Propagation
• Law of Propagation of Uncertainties (LPU):
2
n
 q  2 n n
 q   q 
u (q )   
2
 u ( pi )       u ( pi , p j )
i 1  pi  i 1 j 1  pi   p j 
– where, p are the inputs (components) and q is the output
• E2 Example:
– Uncertainty contributors were analyzed for each of the components
of E2
– Random and systematic effects propagated separately
– Only systematic uncertainties can have correlated effects
• Thickness and width are correlated
2 2 2 2
 E   E   E   E 
v ( E 2 )   2  v T 2 ( P )   2  v T 2 (  2 )   2  v T 2 ( t )   2  vT 2 ( w )
2
 P    2   t   w 
2 2 2 2
 E   E   E   E   E   E 
b2 ( E2 )   2  bT 2 ( P )   2  bT 2 ( 2 )   2  bT 2 (t )   2  bT 2 ( w)  2  2   2  bT (t )bT ( w)
 P    2   t   w   t   w 

University of Florida 9
Mechanical and Aerospace Engineering
Level 3: NIST Component Measurement Uncertainty Table

University of Florida 10
Mechanical and Aerospace Engineering
 Level 4: Contributors of Component Uncertainty
(Further Analysis)
• Numerous different methods to analysis the significance of
uncertainty contributors
• It is important to use the appropriate analysis method
depending on the design of experiments (DOE)
– Either design the experiments properly or match the corresponding
method to the data you already have
• Most DOEs fall into one of these two categories:
Crossed Nested

Same patients in each hospital. Patients unique to each hospital.

University of Florida 11
Mechanical and Aerospace Engineering
 Crossed Design: ANOVA
• Crossed (or factorial) DOEs correspond to analysis of
variance (ANOVA)
• Consider thickness in the E2 example
– Since the SAME specimens were measured in the SAME positions
with the SAME users, then the factors were crossed
– 3-way ANOVA with crossed, random variables was conducted

Nominal: 0.09 x 1 in.
Uncertainty contributors:
Specimen – variability from specimen
to specimen
Position – variation across
measurement surface
User – error from user technique
Measurement repeatability

University of Florida 12
Mechanical and Aerospace Engineering
 Thickness ANOVA
• 3-way ANOVA of crossed, random variables
– Statistical software available for ease of use:
Excel for 2 factors or SAS for 3 or more
• Factors:
– A = specimen a=4 Hypothesis Test for A:
– B = position b=3 H 0 :  2  0
– C = user c=4 H a :  2  0
– Repetitions: n=3
• ANOVA model:
yijkl    i   j   k  ( )ij  ( )ik  (  ) jk  ( )ijk   ijkl

• ANOVA results were not directly used in uncertainty analysis, but were
used to identify significant contributors and validate uncertainty
estimates

University of Florida 13
Mechanical and Aerospace Engineering
 Results: Thickness ANOVA

• Use ANOVA to deterimine the

significance
~
of the
~
contributors
~

of uncertainty in thickness
• Position is most significant
factor with p-value = 0.006
• Not as interested in interactions
in this study
• Used to validate estimated
range of uncertainties of
thickness and width

University of Florida 14
Mechanical and Aerospace Engineering
 Nested Design: VCA
• Nested DOEs correspond to variance component analysis (VCA)
• Consider a two-stage nested design of one specimen for thickness
– Relevant if positions and users were unique each time
– Specimens considered individually since the thickness does not have to be
the same from one specimen to the next
– Data was organized according to position
– y1, y2, and y3 refer to the repeated basic measurements

University of Florida 15
Mechanical and Aerospace Engineering
 Variance Component Analysis of Thickness
• Goal: Develop a nested design to determine the
contribution of each factor in the overall variance
 process
2
   i2 where, i is a component in the process
i

• Variance of the measurement process for one specimen

 mp   bm   u   p
2 2 2 2

– Position – the three locations on the specimen where the

thickness was measured (unique to each specimen)
– User – four different users per position performed the
measurements
– Basic Measurement – three repeated measurements by each user
at each position
• Compare the weight of each contributor to determine
significance

University of Florida 16
Mechanical and Aerospace Engineering
 Concluding Remarks
• Using proper statistical terminology and representation is
necessary to have meaningful results
• You can say your results are “pretty good”, but give what
your definition of “pretty good” is!
• Depending on the project, more or less uncertainty analysis
may be required
• It is important to design your experiments with the
statistical analysis in mind
• Age-old question: How many measurements do I need?
– Obviously depends on the circumstances, so there is no straight
forward answer
– Best recommendation: Feel comfortable enough with your results
that you can predict the next measurement within a desired range

University of Florida 17
Mechanical and Aerospace Engineering