Journal of

Verification,
Validation and
Uncertainty
Quantification
Guest Editorial

Introduction: The 2014 Sandia Verification and Validation Challenge Workshop

Downloaded from http://asmedigitalcollection.asme.org/verification/article-pdf/1/1/010301/6380197/vvuq_001_01_010301.pdf by guest on 07 October 2023

Introduction and they concluded that a healthy diversity of opinions would per-
sist in the community [25].
Over the past 20 years, Sandia National Laboratories (Sandia)
The 2006 Validation Challenge Workshop posed problems with
and ASME [1–6] have played major roles in the development of
a hierarchical flow of information (i.e., materials level calibration,
V&V concepts, methods, and processes by building on the work
which informs validation at an “exploratory” level and
of many other individuals and organizations [7–12]. Along the
“accreditation” level) [26–30]. The hierarchy is a visualization of
way, Sandia has organized workshops to demonstrate the state-of-
how data and model predictions are used during multiple V&V
the-art and highlight particular V&V issues. The most recent, the
analyses [31]. It describes at a high level how the data and models
2014 Sandia V&V Challenge Workshop, was held at the third
relate to each other. This forms the basis for the selection and exe-
ASME V&V Symposium in Las Vegas, NV, on May 8, 2014. As
cution of V&V analyses as part of a larger strategy to improve
a challenge workshop, it was built around a challenge problem
predictions or assess credibility. The participants in the 2006
crafted to exercise specific ideas and spark discussions. The prob-
workshop had to apply multiple V&V methods across a predeter-
lem was released to the community prior to the workshop to allow
mined hierarchy, to make a final prediction and satisfy a regula-
participants time to prepare their responses. Challenge problems
tory requirement. These challenge problems illustrated that (1)
provide a common basis upon which a diverse group of partici-
uncertainty is pervasive throughout modeling work and V&V, (2)
pants can demonstrate methods and ideas and the workshops have
quantitative methods are needed to assess the uncertainty across
become a benchmark of sorts for the V&V community. The 2014
the hierarchy, and (3) a wide range of predictions and uncertainty
workshop focused on the overall V&V strategies to address the
estimates should be expected—even when the hierarchical infor-
challenge problem and also fostered discussion about how a V&V
mation flow is specified [32–35]. Several other UQ and V&V
strategy can establish or assess the credibility of a specific
workshops have focused on methods and theory [36], including
prediction.
some that did not contain a challenge problem [11,12,37].
This special edition of the ASME Journal of Verification, Vali-
By 2012, the workshop organizers felt that adequate attention
dation and Uncertainty Quantification documents the workshop
was being paid to research on V&V theory and methods, and
and is organized as follows. This introductory paper discusses the
adoption of V&V methods and principles was spreading rapidly.
history of workshops and how the goals have evolved with the
Many journals from diverse engineering disciplines were publish-
V&V community—leading to the 2014 workshop. The next paper
ing V&V-focused papers and the topic was popular at
describes the problem statement and how the data provided for the
conferences—notably the ASME V&V Symposium. Both aca-
problem were created (the “truth model”) [13]. The full problem
demia and industry were researching and applying V&V methods,
description, posed to the participants in 2013, is available as a
however, very little was known about how to utilize the results.
technical report [14] but does not include the truth model. These
Decision makers and customers were struggling to understand and
two papers are followed by five contributions that describe the
interpret the evidence gathered from V&V methods. Unfortu-
participants’ responses to the challenge problem [15–19]. Two
nately, the two ends of this spectrum (research versus applica-
discussion papers are also included, in which the authors do not
tions) are not easily connected for several reasons: (1) the V&V
solve the problem but instead address relevant V&V topics while
community of practice is relatively small, (2) a huge range of
using the challenge problem as context [20,21]. The special edi-
expertise is required to perform all the V&V activities, (3) the
tion concludes with a summary paper that describes the responses
incentives of researchers (novel methods and skills, broad and
and lessons learned [22].
general impact, and publishing ideas) and practicing engineers
(delivering results, creating a process tailored to one organization,
A History of Challenge Workshops and maintaining competitive advantage) have limited overlap, and
The 2002 Workshop on Alternative Representations of Episte- (4) the benefits of V&V are hard to measure and communicate.
mic Uncertainty dealt with uncertainty quantification (UQ), or The organizers saw the need to continue developing V&V meth-
how to characterize, model, and propagate uncertainty with both ods, provide the community with V&V experience, and investi-
epistemic and aleatoric nature [23,24]. The workshop highlighted gate the decision-making process. In response, they launched
the importance of uncertainty, and the need to quantify uncer- efforts to develop a new problem and hold a third challenge
tainty and understand its impact on model predictions. This work- workshop.
shop’s goal was to compare different concepts and algorithms to
describe predictions that rely on uncertain information. The par- The 2014 Sandia Verification and Validation Challenge
ticipants found that each method had advantages and drawbacks, Workshop
The 2014 Sandia Verification and Validation Challenge Work-
The United States Government retains, and by accepting the article for shop was a daylong event with ten talks and follow-up discus-
publication, the publisher acknowledges that the United States Government retains, a
nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the
sions. After the organizers opened the workshop, nine participants
published form of this work, or allow others to do so, for United States government presented responses to the challenge problem or their views on the
purposes. role of V&V in decision support, followed by an open discussion.

Journal of Verification, Validation and Uncertainty Quantification MARCH 2016, Vol. 1 / 010301-1
C 2016 by ASME
Copyright V
The participants were from academia, consulting firms, and the
U.S. research laboratories, and were predominantly engineers and
mathematicians. The workshop was open to all attendees of the
ASME V&V Symposium and an estimated 100 people attended
some or all of the talks and discussion.
Hu (Sandia National Laboratories) served as workshop modera-
tor and introduced the challenge problem [38]. This was followed
by six responses from: Shields et al. (John Hopkins University)
[39,40], Roy et al. (Virginia Tech) [41], Chen et al. (Northwestern
University) [42], Xi and Pan (University of Michigan—Dearborn)
[43], Mahadevan and Mullins (Vanderbilt) [44], and Beghini and
Hough (Sandia National Laboratories) [45]. For brevity, only the
team leads are listed but each presentation was the work of a
group. Three more speakers presented their thoughts on how
V&V plays a role in decision making, Paez (consultant) gave an
economic analysis of whether modeling, simulation, and V&V are
worth the cost [46], Elele (U.S. Naval Air Warfare Center) dis-
cussed how V&V can be used to assess risk levels and impact on
projects [47], and Brodrick (U.S. Naval Surface Warfare Center)
discussed how models are currently used to support qualification
activities, and a vision for how V&V and UQ should be incorpo-
rated into that process [48]. After the workshop, several of the
participants continued to refine their solutions and discussion
topics, which led to the creation of this special edition.
The Challenge Problem
The scenario described in the challenge problem is an engineer-
ing project to predict the behavior of a population of simple stor-
age tanks under a variety of loads. Various experimental studies
have been completed, and limited data are available. In addition, a
finite-element model has been developed to predict structural
response—but not mechanical failure. The scenario ends in a deci-
sion whether the entire population should stay in service or needs
to be replaced; however, the challenge is simplified into three
tasks: predict probability of failure plus an uncertainty estimate,
assess the credibility of the prediction, and describe the V&V
strategy used. This paper reveals the motivations and the impor-
tant features; the problem statement is described in the following
papers [13,14].
The challenge problems must trade-off the flexibility to allow a
variety of V&V approaches and the constraints required to focus
the participants’ attention. This problem de-emphasizes individual
V&V methodologies and focuses on strategies to gather and inte-
grate V&V evidence. This was accomplished by defining an open-
ended set of objectives—in contrast to the more prescriptive nature
of previous challenge problems [24,28–30,36]. The principle chal-
lenge was to develop a defensible V&V strategy, which prioritized
which analyses to complete and which methods to use. The partici-
pants were free to choose how to use the data and model to achieve
the ultimate prediction and had the opportunity—but not the
obligation—to perform many analyses: UQ, calibration, solution
verification, validation, etc. In addition, participants had to inte-
grate the results and communicate their conclusions. This parallels
the projects tackled by engineering organizations; the purpose is to
supply credible advice to support technical decisions and there is no
set of instructions to follow. This focus on V&V strategy is a small
step toward bridging the gap from theory and methods to the proc-
esses needed to support decisions. The question is now: Which V&V
analyses are the appropriate analyses to perform, in order to make the
desired predictions? The next question will be: How do these analyses
support and impact the ultimate decision?
To successfully investigate these questions about V&V strat-
egies, the challenge problem required a great deal of complexity
and context. The scope of the problem enabled participants to con-
struct their own hierarchy based on the available data and models.
This also raised the question of how to aggregate or integrate
uncertainty estimates from different analyses—for example,
uncertainty from a mesh convergence study, uncertainty and vari-
ability of material properties, plus the effects of measurement
010301-2 / Vol. 1, MARCH 2016
error. This requires careful considerations of the “meaning” of
each uncertainty and the implications for credibility, in addition to
methods for UQ.
The large scope also required a large commitment from the par-
ticipants, and the expectation was that all participants would be
limited by time, resources, and expertise—it is a major challenge
to assemble a team that includes experts in verification, validation,
UQ, physics domains, and systems engineering. That being said,
several compromises were made to ensure the problem could be
solved in a reasonable timeframe.
 The end decision of whether to replace the tanks is not part of
the challenge. The problem lacks the required context, such as
cost of the tanks, the state of the company, risk tolerances, etc.
Instead, an arbitrary criterion is created for the probability of
failure. As a consequence, participants focused on a specific
Downloaded from http://asmedigitalcollection.asme.org/verification/article-pdf/1/1/010301/6380197/vvuq_001_01_010301.pdf by guest on 07 October 2023
prediction and did not have to speculate on the intended use of
the model. The participants and audience were expected to be
V&V analysts and not systems engineers or decision makers,
so this challenge was limited to fit their backgrounds. Never-
theless, the additional context was included to raise awareness
of how V&V eventually impacts decisions.
 The data, model, and code were supplied. This removed all
choices about model development and design of experiments,
but greatly reduced the workload. Four finite-element mesh
resolutions were available, based on the same geometry, which
could be solved using the provided code. The code was
described as a finite-element solver, but was actually an alge-
braic function. This enabled anyone to easily and cheaply run
the code, but the model behavior was not the same as a finite-
element model—and code verification was not sensible. The
data were created from a truth model—not physical
experiments.
 Only a single structural mechanics example was provided. The
organizers wanted a compelling engineering problem rather
than a generic mathematical exercise, but needed to keep it
simple and accessible for engineers from any physics disci-
pline background. Inevitably, some domain knowledge was
helpful in interpreting the results.
The challenge workshop was not a competition and there is no
“best” strategy. The responses were not scored or ranked. In fact,
the open-ended problem formulation meant that direct comparison
of responses was impossible. Discrepancies could not be traced to
a single cause because so many different assumptions and method
choices were made by each team. Also, no limits were placed on
the time invested in the problem, and the participating teams had
a wide range of effort levels and priorities. The only step taken to
ensure fairness was to provide all the teams access to the same
information and instructions. The organizers did not collaborate
with any of the participants or reveal the source of the data prior
to this special edition. The sources of the data, the truth model,
are now available [13]. Again, the organizers stress that scoring
the approaches by comparing to the “truth” is not the intention of
this activity—indeed, it is not clearly understood how to objec-
tively judge any of the approaches at all. The organizers are not
interested in the “correct result”; the goal is to learn about differ-
ent V&V strategies.
Conclusions
Sandia has used challenge workshops as benchmarks of the
V&V community. The community has evolved from developing
individual UQ and V&V methods, for investigating how to inte-
grate analyses across a hierarchy and now designing the appropri-
ate hierarchy. Development and demonstration of individual
methods are still absolutely critical, but this workshop emphasized
V&V strategy as the current challenge. The quality of the work-
shop responses and the discussions at the ASME V&V Sympo-
sium indicate that the V&V community is growing, and V&V
Transactions of the ASME
theory and methods are maturing. Again, the workshop saw a
healthy diversity of V&V strategies, which reflect the different
priorities and backgrounds of the participants. A summary of the
responses and commentary is included in the following paper
[22]. The major lesson is that we need more experience with a
larger variety of case studies and we need to learn how to judge
the efficacies of V&V strategies.
Looking forward, a major obstacle will be the vast scope of
V&V, which relates to the fields of statistics, probability theory,
numerical algorithms, systems engineering, and many intermedi-
ate topics. It is difficult to merge quantitative and rigorous meth-
ods with the qualitative and subjective aspects of credibility. The
next challenge may concern the role of V&V with respect to mod-
eling and simulation and the decision-making process.
The workshop organizers hope that the V&V community will
build upon the work in this special edition and make use of this
challenge problem (and those from prior workshops). These prob-
lems are excellent case studies for teaching or demonstrating
V&V approaches and concepts and will spark discussions about
the role of V&V in engineering work.
Acknowledgment
This challenge problem and the resulting workshop were made
possible with support from the Sandia National Laboratories and
ASME. We wish to thank Greg Weirs, Laura Swiler, Walt Wit-
kowski, and the Dakota team at the Sandia National Laboratories,
and Ryan Crane and the V&V Standards committees at ASME for
helping make the workshop a success. We are especially grateful
to the workshop participants, who made this special edition possi-
ble, including those whose work does not appear in this issue:
Michael Shields, Thomas Brodrick, and James Elele. Sandia
National Laboratories is a multiprogram laboratory managed and
operated by the Sandia Corporation, a wholly owned subsidiary of
the Lockheed Martin Corporation, for the U.S. Department
of Energy’s National Nuclear Security Administration under
Contract No. DE-AC04-94AL85000.
References
[1] Oberkampf, W. L., Pilch, M., and Trucano, T. G., 2007, “Predictive Capabil-
ity Maturity Model for Computational Modeling and Simulation,” Sandia
National Laboratories, Report No. SAND2007-5948.
[2] Trucano, T. G., Pilch, M., and Oberkampf, W. L., 2002, “General Concepts
for Experimental Validation of ASCI Code Applications,” Sandia National
Laboratories, Report No. SAND2002-0341.
[3] Trucano, T. G., Pilch, M., and Oberkampf, W. L., 2003, “On the Role of
Code Comparisons in Verification and Validation,” Sandia National Labora-
tories, Report No. SAND2003-2752.
[4] ASME V&V 20, 2009, Standard for Verification and Validation in Computa-
tional Fluids and Heat Transfer, The American Society of Mechanical Engi-
neers, New York.
[5] ASME V&V 10-2006, 2006, Guide for Verification and Validation in Com-
putational Solid Mechanics, The American Society of Mechanical Engineers,
New York.
[6] Oberkampf, W. L., and Barone, M. F., 2006, “Measures of Agreement
Between Computation and Experiment: Validation Metrics,” J. Comput.
Phys., 217(1), pp. 5–36.
[7] Oberkampf, W. L., 1998, “Bibliography for Verification and Validation in
Computational Simulation,” Sandia National Laboratories, Report No.
SAND98-2041.
[8] Schwer, L. E., and Oberkampf, W. L., 2007, “Special Issue on Verification
and Validation,” Eng. Comput., 23(4), pp. 243–244.
[9] Oberkampf, W. L., Sindir, M., and Conlisk, A. T., 1998, “Guide for the Veri-
fication and Validation of Computational Fluid Dynamics Simulations,”
AIAA Paper No. G-077-1998.
[10] Nitta, C. K., and Logan, R. W., 2004, “ASCI V&V at LLNL: An Unclassified
Bibliography,” Lawrence Livermore National Laboratory, Report No.
UCRL-AR-203864.
[11] Pace, D., 2002, “Foundations’02 Overview,” Foundations’02 a Workshop on
Model and Simulation Verification and Validation for the 21st Century, D.
Pace, ed., JHU/APL, Laurel, MD.
[12] Youngblood, S. M., 2004, “Roadmap for VV&A Technology Advancement,”
Foundations’04: A Workshop for V&V in the 21st Century, Arizona State
University, Phoenix, AZ, Defense Modeling & Simulation Office.
Journal of Verification, Validation and Uncertainty Quantification
[13] Hu, K. T., and Orient, G. E., 2016, “The 2014 Sandia V&V Challenge: Prob-
lem Statement,” ASME J. Verif., Validation, Uncertainty Quantif., 1(1), p.
011001.
[14] Hu, K. T., 2013, “2014 V&V Challenge: Problem Statement,” Sandia
National Laboratories, Report No. SAND2013-10486P.
[15] Choudhary, A., Voyles, I. T., Roy, C. J., Oberkampf, W. L., and Patil, M.,
2016, “Probability Bounds Analysis Applied to the Sandia Verification and
Validation Challenge Problem,” ASME J. Verif., Validation, Uncertainty
Quantif., 1(1), p. 011003.
[16] Li, W., Chen, S., Jiang, Z., Apley, D. W., Lu, Z., and Chen, W., 2016,
“Integrating Bayesian Calibration, Bias Correction, and Machine Learning
for the 2014 Sandia Verification and Validation Challenge Problem,” ASME
J. Verif., Validation, Uncertainty Quantif., 1(1), p. 011004.
[17] Mullins, J., and Mahadevan, S., 2016, “Bayesian Information Fusion for
Model Calibration, Validation, and Prediction,” ASME J. Verif., Validation,
Uncertainty Quantif., 1(1), p. 011006.
[18] Beghini, L. L., and Hough, P. D., 2016, “Sandia V&V Challenge Problem: A
PCMM-Based Approach to Assessing Prediction Credibility,” ASME J.
Verif., Validation, Uncertainty Quantif., 1(1), p. 011002.
[19] Xi, Z., and Yang, R. J., 2016, “Reliability Analysis With Model Uncertainty
Downloaded from http://asmedigitalcollection.asme.org/verification/article-pdf/1/1/010301/6380197/vvuq_001_01_010301.pdf by guest on 07 October 2023
Coupling With Parameter and Experimental Uncertainties: A Case Study of
2014 V&V Challenge Problem,” ASME J. Verif., Validation, Uncertainty
Quantif., 1(1), p. 011005.
[20] Hu, K. T., and Paez, T., 2016, “Why Do Verification and Validation?”
ASME J. Verif., Validation, Uncertainty Quantif., 1(1), p. 011008.
[21] Paez, P. J., Paez, T., and Hasselman, T. K., 2016, “Economic Analysis of
Model Validation for a Challenge Problem,” ASME J. Verif., Validation,
Uncertainty Quantif., 1(1), p. 011007.
[22] Schroeder, B. B., Hu, K. T., Mullins, J. G., and Winokur, J. G., 2016,
“Summary of the 2014 Sandia V&V Challenge Workshop,” ASME J. Verif.,
Validation, Uncertainty Quantif., 1(1), p. 015501.
[23] Helton, J. C., and Oberkampf, W. L., 2004, “Alternative Representations
of Epistemic Uncertainty,” Reliab. Eng. Syst. Saf., 85(1–3), pp. 1–10.
[24] Oberkampf, W. L., Helton, J. C., Joslyn, C. A., Wojtkiewicz, S. F.,
and Ferson, S., 2004, “Challenge Problems: Uncertainty in System
Response Given Uncertain Parameters,” Reliab. Eng. Syst. Saf., 85(1–3),
pp. 11–19.
[25] Ferson, S., Joslyn, C. A., Helton, J. C., Oberkampf, W. L., and Sentz, K.,
2004, “Summary From the Epistemic Uncertainty Workshop: Consensus
Amid Diversity,” Reliab. Eng. Syst. Saf., 85(1–3), pp. 355–369.
[26] Pilch, M., 2008, “Preface,” Comput. Methods Appl. Mech. Eng.,
197(29–32), pp. 2373–2374.
[27] Hills, R. G., Pilch, M., Dowding, K. J., Red-Horse, J., Paez, T. L., Babuska,
I., and Tempone, R., 2008, “Validation Challenge Workshop,” Comput.
Methods Appl. Mech. Eng., 197(29–32), pp. 2375–2380.
[28] Babuska, I., Nobile, F., and Tempone, R., 2008, “Formulation of the Static
Frame Problem,” Comput. Methods Appl. Mech. Eng., 197(29–32),
pp. 2496–2499.
[29] Dowding, K. J., Pilch, M., and Hills, R. G., 2008, “Formulation of the
Thermal Problem,” Comput. Methods Appl. Mech. Eng., 197(29–32), pp.
2385–2389.
[30] Red-Horse, J. R., and Paez, T. L., 2008, “Sandia National Laboratories Vali-
dation Workshop: Structural Dynamics Application,” Comput. Methods
Appl. Mech. Eng., 197(29–32), pp. 2578–2584.
[31] Oberkampf, W. L., and Trucano, T. G., 2002, “Verification and Validation
in omputational Fluid Dynamics,” Prog. Aerosp. Sci., 38(3), pp. 209–272.
[32] Dowding, K. J., Red-Horse, J., Hills, R. G., Paez, T. L., Babuska, I., and
Tempone, R., 2008, “Validation Challenge Workshop Summary,” Comput.
Methods Appl. Mech. Eng., 197(29–32), pp. 2381–2384.
[33] Paez, T. L., and Red-Horse, J. R., 2008, “Structural Dynamics Challenge
Problem: Summary,” Comput. Methods Appl. Mech. Eng., 197(29–32),
pp. 2660–2665.
[34] Hills, R. G., Dowding, K. J., and Swiler, L., 2008, “Thermal Challenge
Problem: Summary,” Comput. Methods Appl. Mech. Eng., 197(29–32),
pp. 2490–2495.
[35] Babuska, I., and Tempone, R., 2008, “Static Frame Challenge Problem:
Summary,” Comput. Methods Appl. Mech. Eng., 197(29–32),
pp. 2572–2577.
[36] Luis, G. C., Sean, P. K., and Daniel, P. G., 2014, “The NASA Langley
Multidisciplinary Uncertainty Quantification Challenge,” 16th AIAA Non-
Deterministic Approaches Conference, American Institute of Aeronautics
and Astronautics.
[37] Webster, C., Collis, S., Gunzburger, M., Trucano, T., and Womble, D., 2015,
“Sandia CSRI Workshop on Mathematical Methods for Validation and Ver-
ification,” Last accessed Oct. 26, 2015, http://www.cs.sandia.gov/CSRI/
Workshops/2007/MMVV/index.html
[38] Hu, K. T., 2014, “The Sandia National Laboratories 2014 Verification &
Validation Challenge Workshop,” ASME Paper No. V&V2014-7211.
[39] Shields, M. D., Teferra, K., and Kim, H., 2014, “V&V Challenge Problem:
An Efficient Monte Carlo Method Incorporating the Effects of Model Error,”
ASME Paper No. V&V2014-7214.
[40] Shields, M. D., Teferra, K., Hapij, A., and Daddazio, R. P., 2015, “Refined
Stratified Sampling for Efficient Monte Carlo Based Uncertainty
Quantification,” Reliab. Eng. Syst. Saf., 142, pp. 310–325.
[41] Roy, C., Choudhary, A., Voyles, I. T., and Patil, M., 2014, “Probability
Bounds Analysis Applied to the Sandia V&V Challenge Problem,” ASME
Paper No. V&V2014-7219.
MARCH 2016, Vol. 1 / 010301-3
 [42] Chen, W., Apley, D., Li, W., Jiang, Z., and Chen, S., 2014, “A Bayesian
Inference Approach to the Validation Challenge Problem,” ASME Paper No. Kenneth T. Hu
V&V2014-7216. Mem. ASME
[43] Xi, Z., and Pan, H., 2014, “Copula-Based Validation Approach for 2014 V&V, UQ, and Credibility Processes Department,
V&V Challenge Problem,” ASME Paper No. V&V2014-7220. Sandia National Laboratories,
[44] Mahadevan, S., and Mullins, J., 2014, “Bayesian Information Fusion
for Model Validation and Prediction,” ASME Paper No. V&V2014-7213. P.O. Box 5800, MS 0828,
[45] Beghini, L. L., and Hough, P., 2014, “Surrogate-Based V&V,” ASME Paper Albuquerque, NM 87185-0828
No. V&V2014-7226.
[46] Paez, P. J., Paez, T., and Hasselman, T. K., 2014, “The Economics of Model Brian Carnes
Validation,” ASME Paper No. V&V2014-7215.
[47] Elele, J., and Hall, D., 2014, “Top-Down Risk Based Approach to M&S Mem. ASME
VV&A,” ASME Paper No. V&V2014-7210. V&V, UQ, and Credibility Processes Department,
[48] Brodrick, T., 2014, “Sample Problem From a DOD Perspective,” ASME Sandia National Laboratories,
Paper No. V&V2014-7217. P.O. Box 5800, MS 0828,
Albuquerque, NM 87185-0828

Vicente Romero
Mem. ASME

Downloaded from http://asmedigitalcollection.asme.org/verification/article-pdf/1/1/010301/6380197/vvuq_001_01_010301.pdf by guest on 07 October 2023

V&V, UQ, and Credibility Processes Department,
Sandia National Laboratories,
P.O. Box 5800, MS 0828,
Albuquerque, NM 87185-0828

010301-4 / Vol. 1, MARCH 2016 Transactions of the ASME