Integrating Physics-Based Modeling with Machine Learning: A

Survey

基于物理的建模与机器学习的集成：综述

Abstract

摘要

In this manuscript, we provide a structured and comprehensive overview of techniques to

integrate machine learning with physics-based modeling. First, we provide a summary of
application areas for which these approaches have been applied. Then, we describe classes of
methodologies used to construct physics-guided machine learning models and hybrid physics-
machine learning frameworks from a machine learning standpoint. With this foundation, we then
provide a systematic organization of these existing techniques and discuss ideas for future
research.

在这篇文章中，我们对机器学习与基于物理的建模相结合的技术进行了结构化和全面的概
述。
首先，我们总结了应用这些方法的应用领域。
然后，我们从机器学习的角度描述了用于构建物理引导机器学习模型和混合物理机器学习
框架的方法。在此基础上，我们对这些现有技术进行了系统的组织，并讨论了未来研究的
思路。

1 Introduction
1 引言

Machine learning (ML) models, which have already found tremendous success in commercial
applications, are beginning to play an important role in advancing scientific discovery in domains
traditionally dominated by physics-based models [Karpatne et al., 2017a]. The use of ML models
is particularly promising in scientific problems involving processes that are not completely
understood, or where it is computationally infeasible to run physics-based models at desired
resolutions in space and time. However, the application of even the state-of-the-art black box ML
models has often met with limited success in scientific domains due to their large data
requirements, inability to produce physically consistent results, and their lack of generalizability
to out of sample scenarios. Given that neither an ML-only nor a scientific knowledge-only
approach can be considered sufficient for complex scientific and engineering applications, the
research community is beginning to explore the continuum between physics-based and ML
models, where both scientific knowledge and data are integrated in a synergistic manner.
机器学习（ML）模型已在商业应用中取得巨大成功，在传统上以基于物理的模型为主的领
域中，机器学习模型开始在推动科学发现方面发挥重要作用[Karpatne 等人，2017a]。ML 模
型的使用在涉及未完全理解的过程的科学问题中尤其有前景，或者在计算上无法以所需的
空间和时间分辨率运行基于物理的模型的情况下。然而，即使是最先进的黑盒 ML 模型在
科学领域的应用也常常遇到有限的成功，因为它们需要大量的数据，无法产生物理上一致
的结果，并且缺乏对样本外场景的通用性。鉴于仅 ML 和仅科学知识的方法都不足以满足
复杂的科学和工程应用，研究界开始探索基于物理和 ML 模型之间的连续体，其中科学知
识和数据以协同方式集成。

This paradigm is fundamentally different from mainstream practices in the ML community for
making use of domainspecific knowledge, albeit in subservient roles, e.g., feature engineering or
post-processing. In contrast to these practices that can only work with simpler forms of heuristics
and constraints, this approach explores a deeper coupling of ML methods with scientific
knowledge.

这种范式与 ML 社区中利用领域特定知识的主流实践有根本不同，尽管是在从属角色中，
例如特征工程或后处理。与这些只能使用更简单形式的启发式和约束的实践不同，这种方
法探索了 ML 方法与科学知识的更深层耦合。

Even though the idea of integrating physics and ML models has picked up momentum just in the
last few years, there is already a vast amount of work on this topic. This work is being pursued in
diverse disciplines (e.g., climate science, turbulence modeling, material discovery, quantum
chemistry, biomedical science, biomarker discovery, and hydrology).

尽管在过去几年中，将物理和 ML 模型相结合的想法已经初见端倪，但在这个主题上已经
有了大量的工作。这项工作正在多个学科中进行（例如，气候科学、湍流建模、材料发现
量子化学、生物医学科学、生物标志物发现和水文学）。

Also, it is being performed in the context of diverse objectives specific to these applications. Early
results in isolated and relatively simple scenarios have been promising, and the expectations are
rising for this paradigm to accelerate scientific discovery and help address some of the biggest
challenges that are facing humanity in terms of climate, health, and food security.

此外，它是在特定于这些应用程序的不同目标的背景下执行的。在孤立和相对简单的场景
中的早期结果是有希望的，人们对这种范式的期望正在上升，以加速科学发现，并帮助解
决人类在气候、健康和粮食安全方面面临的一些最大挑战。

The goal of this survey is to bring these exciting developments to the ML community, to make
them aware of the progress that has been made, and the gaps and opportunities that exist for
advancing research in this promising direction.

本次调查的目的是将这些激动人心的发展带给 ML 社区，让他们了解已经取得的进展，以
及在这一有希望的方向上推进研究所存在的差距和机会。
To enable this, we organize the paper as follows. Section 2 describes different objectives that can
be achieved through combinations of physics and machine learning. Then, Section 3 discusses ML
methods that researchers are using to achieve these objectives. Table 1 categorizes the work
surveyed in this paper by objective and approach. We hope that this survey will also be valuable
for physical scientists who are interested in exploring the use of ML to enhance modeling in their
respective disciplines. Please note that while the focus of this survey is largely on physics-based
models, the ideas carry over to a wide range of scientific disciplines.

为了实现这一点，我们将论文组织如下。第 2 节描述了通过物理和机器学习的结合可以实
现的不同目标。然后，第 3 节讨论了研究人员用于实现这些目标的 ML 方法。表 1 按目标和
方法对本文调查的工作进行了分类。我们希望这项调查对那些有兴趣探索 ML 在各自学科
中的使用以增强建模的物理科学家也有价值。请注意，虽然本次调查的重点主要是基于物
理的模型，但这些想法会延伸到广泛的科学学科中。

2 Objectives of Physics-ML Integration

物理 ML 集成的 2 个目标

2.1 Improving predictions beyond that of state-of-the-art physical models

2.1 改进超越最先进物理模型的预测

Physics-based models are used extensively to model engineering and environmental systems.
Even though these models are based on known physical laws, in most cases, they are necessarily
approximations of reality due to incomplete knowledge of certain processes, which introduces
bias. In addition, they often contain a large number of parameters whose values must be
estimated with the help of limited observed data, degrading their performance further, especially
due to heterogeneity in the underlying processes in both space and time. The limitations of
physics-based models cut across discipline boundaries and are well known in the scientific

基于物理的模型广泛用于工程和环境系统的建模。尽管这些模型基于已知的物理定律，但
在大多数情况下，由于对某些过程的不完全了解，它们必然是对现实的近似，这会引入偏
差。此外，它们通常包含大量参数，这些参数的值必须借助有限的观察数据进行估计，这
进一步降低了它们的性能，尤其是由于基础过程在空间和时间上的异构性。基于物理的模
型的局限性跨越了学科边界，在科学界是众所周知的

Given that ML models can automatically extract complex relationships from data, they appear
promising for scientific problems with physical processes that are not fully understood by
researchers, but for which data of adequate quality and quantity is available. However, the notion
of black-box application of ML has met with limited success in scientific domains since ML
approaches are prone to false discoveries due to non-stationary relationships among the
variables that are difficult to capture from limited observation data. Hence, there is an interest in
combining elements of physics-based modeling with state-of-the-art ML models to leverage their
complementary strengths. The objective is to develop innovative integrated physics-ML models
to better capture the dynamics in scientific systems and advance the understanding of underlying
physical processes. Effective representation of physical processes requires the development of
novel abstractions and architectures that can simulate these processes that may be evolving and
interacting at multiple scales.

由于 ML 模型可以自动从数据中提取复杂的关系，因此对于研究人员尚未完全理解但有足
够质量和数量的数据可用的物理过程的科学问题，ML 模型似乎很有希望。然而，由于难以
从有限的观测数据中捕捉变量之间的非平稳关系，ML 方法容易出现错误发现，因此 ML 的
黑箱应用概念在科学领域的成功有限。因此，人们有兴趣将基于物理的建模元素与最先进
的 ML 模型相结合，以利用它们的互补优势。目标是开发创新的综合物理 ML 模型，以更好
地捕捉科学系统中的动态，并促进对潜在物理过程的理解。物理过程的有效表示需要开发
新的抽象和架构，以模拟这些可能在多个尺度上进化和交互的过程。

2.2 Parameterization
2.2 参数化

Complex physics-based models (e.g., simulations in climate, weather, turbulence modeling,

astrophysics) are often capable of capturing physical reality more precisely than simpler models,
as they may simulate physical processes at higher spatial and temporal resolution (e.g., cloud
physics in climate models). However, they often require a prohibitively high computational cost to
run. In such cases, specific complex dynamical processes are replaced by simplified physical
approximations that are described by parameters, whose values are estimated from observed
data. This is known as parameterization.

基于物理的复杂模型（例如气候、天气、湍流建模、天体物理学中的模拟）通常能够比简
单模型更精确地捕捉物理现实，因为它们可能以更高的时空分辨率模拟物理过程（例如气
候模型中的云物理）。然而，它们通常需要极高的计算成本才能运行。在这种情况下，特
定的复杂动力学过程被由参数描述的简化物理近似所取代，这些参数的值是根据观察数据
估计的。这称为参数化。

The failure to correctly identify parameter values can make the model less robust when
generalized to different scenarios. Errors that result from imperfect parameterization can also
feed into other components of the entire physics-based model and deteriorate the modeling of
important physical processes. Hence, there is a great interest in ML models to learn new
parameterizations directly from high-resolution model simulations [Schneider et al., 2017].
Specifically, the ML model is trained on a high-resolution model in which the physical process
desired is resolved and can be substituted into a coarser resolution model as a more accurate
parameterization for processes of interest [Goldstein et al., 2014; Chan and Elsheikh, 2017;
Gentine et al., 2018].

当推广到不同场景时，未能正确识别参数值可能会降低模型的鲁棒性。由于参数化不完善
而导致的错误也会影响整个基于物理的模型的其他组件，并恶化重要物理过程的建模。因
此 ， ML 模 型 非 常 有 兴 趣 直 接 从 高 分 辨 率 模 型 模 拟 中 学 习 新 的 参 数 化 [Schneider 等 人 ，
2017]。具体而言，ML 模型是在高分辨率模型上训练的，在该模型中，所需的物理过程被
解析，并且可以替换为更粗分辨率模型，作为感兴趣过程的更精确参数化[Goldstein 等人，
2014；Chan 和 Elsheikh，2017；Gentine 等人，2018]。

2.3 Forward Solving Partial Differential Equations

2.3 正解偏微分方程

In many physical systems, governing equations are known but direct numerical solutions of
partial differential equations (PDEs) using common methods like the Finite Elements Method or
the Finite Difference Method [Fish and Belytschko, 2007] are prohibitively expensive. In such
cases, traditional methods are not ideal or sometimes even possible. A common technique is to
use an ML model as a surrogate for the solution to reduce computation time [Lagaris et al.,
1998]. More recently with the advancement of computational power, neural networks models
have shown success in approximating solutions across different kinds of physicsbased PDEs
[Arsenault et al., 2014; Rudd and Ferrari, 2015; Khoo et al., 2019]. As a step further, deep neural
networks models in particular have shown success in approximating solutions across high
dimensional physics-based PDEs previously inaccessible by ML [Han et al., 2018]. However, slow
convergence in training, limited applicability to many complex systems, and reduced accuracy
due to unawareness of physical laws can prove problematic.

在许多物理系统中，控制方程是已知的，但使用有限元法或有限差分法等常用方法直接求
解偏微分方程（PDE）[Fish 和 Belytschko，2007]的成本过高。在这种情况下，传统方法并
不理想，有时甚至不可能。一种常见的技术是使用 ML 模型作为解决方案的替代物，以减
少计算时间[Lagaris 等人，1998]。最近，随着计算能力的提高，神经网络模型在跨不同种
类 的 基 于 物 理 的 偏 微 分 方 程 近 似 解 方 面 取 得 了 成 功 [Arsenault 等 人 ， 2014 ； Rudd 和
Ferrari，2015；Khoo 等人，2019]。更进一步的是，深度神经网络模型在跨 ML 以前无法实
现的高维基于物理的偏微分方程近似解方面取得了成功[Han 等人，2018]。然而，训练收敛
缓慢，对许多复杂系统的适用性有限，以及由于不了解物理定律而导致的准确性降低，这
些都是有问题的。

2.4 Inverse Modeling

2.4 反建模

The forward modeling of a physical system uses the physical parameters of the system (e.g.,
mass, temperature, charge, physical dimensions or structure) to predict the next state of the
system or its effects (outputs). In contrast, inverse modeling uses the (possibly noisy) output of a
system to infer the physical parameters. A well-known example of inverse modeling is the use of
x-ray images from a CT scan to create a 3D image reflecting the structure of part of a person’s
body.

物理系统的正向建模使用系统的物理参数（例如质量、温度、电荷、物理尺寸或结构）来
预测系统的下一个状态或其影响（输出）。相反，逆建模使用系统的（可能有噪声）输出
来推断物理参数。逆建模的一个众所周知的例子是使用 CT 扫描的 x 射线图像来创建反映人
体部分结构的 3D 图像。

Inverse problems are traditionally solved using regularized regression techniques. Data-driven
methods have seen success in inverse problems in remote sensing of surface properties [Dawson
et al., 1992], photonics [Pilozzi et al., 2018], and medical imaging [Lunz et al., 2018] among many
others. Recently, novel algorithms using deep learning and neural networks have been applied to
inverse problems. While still in their infancy, these techniques exhibit strong performance for
applications like computerized tomography [Chen et al., 2017], seismic processing [V amaraju and
Sen, 2019], or various sparse data problems.

逆问题通常使用正则回归技术求解。数据驱动方法在地表特性遥感反问题方面取得了成功
[Dawson 等人，1992]、光子学[Pilozzi 等人，2018]和医学成像[Lunz 等人，2018]等。最近，
使用深度学习和神经网络的新算法已应用于反问题。虽然这些技术仍处于婴儿期，但在计
算机断层扫描[Chen 等人，2017]、地震处理[V amaraju 和 Sen，2019]或各种稀疏数据问题
等应用中表现出了强大的性能。

However, there are noted challenges in inverse modeling particularly in small sample size and
paucity of ground-truth label scenarios [Karpatne et al., 2018]. The integration of prior physical
knowledge is common in current approaches to the inverse problem, and its integration into
inverse problem ML applications has the potential to improve data efficiency and increase its
ability to solve ill-posed inverse problems.

然而，在逆建模方面存在着明显的挑战，尤其是在小样本量和缺乏地面真值标签的情况下
[Karpatne 等人，2018]。在反问题的当前方法中，先验物理知识的集成是常见的，将其集成
到反问题 ML 应用程序中有可能提高数据效率并提高其解决不适定反问题的能力。

2.5 Discovering Governing Equations

2.5 发现控制方程

When the governing equations of a dynamical system are known, they allow for forecasting,
control, and the analysis of system stability and bifurcations. However, many complex dynamic
systems have no formal analytic descriptions (e.g., neuroscience, cell biology, finance,
epidemiology). Often in these cases, data is abundant but the underlying governing equations
remain elusive.

当动力系统的控制方程已知时，它们允许预测、控制和分析系统稳定性和分岔。然而，许
多复杂的动态系统没有正式的分析描述（例如，神经科学、细胞生物学、金融学、流行病
学）。通常在这些情况下，数据丰富，但基本的控制方程仍然难以捉摸。

Advances in ML for the discovery of these governing equations has become an active research
area with rich potential to integrate principles from applied mathematics and physics with
modern ML methods. Data-driven discovery of governing equations has recently been pioneered
by [Bongard and Lipson, 2007; Schmidt and Lipson, 2009] where they apply symbolic regression
to differences between computed derivatives and analytic derivatives to determine underlying
dynamical systems. More recently, [Brunton et al., 2016; Rudy et al., 2017] uses sparse regression
built on a dictionary of functions and partial derivatives to construct governing equations.
ML 在发现这些控制方程方面的进展已成为一个活跃的研究领域，具有将应用数学和物理原
理 与 现 代 ML 方 法 相 结 合 的 丰 富 潜 力 。 控 制 方 程 的 数 据 驱 动 发 现 最 近 由 [Bongard 和
Lipson，2007；Schmidt 和 Lipson，2009]开创，他们将符号回归应用于计算导数和分析导数
之间的差异，以确定潜在的动力系统。最近，[Brunton 等人，2016；Rudy 等人，2017]使用
基于函数和偏导数字典的稀疏回归来构建控制方程。

3 Physics-ML Methods
3 物理 ML 方法

3.1 Physics-Guided Learning

3.1 物理引导学习

Scientific problems often exhibit a high degree of complexity due to relationships between many
physical variables varying across space and time at different scales simultaneously.

由于许多物理变量之间的关系在不同的时间和空间尺度上同时变化，科学问题往往表现出
高度的复杂性。

Standard ML models can fail to capture such relationships directly from data, especially when
provided with limited observation data. This is one reason for their failure to generalize to
scenarios not encountered in training data. In the following, we discuss a number of ways
researchers are beginning to incorporate physical knowledge into the learning process such that
ML models can capture generalizable dynamic patterns consistent with established physical laws.

标准 ML 模型可能无法直接从数据中捕捉此类关系，尤其是当提供的观测数据有限时。这
是他们未能推广到训练数据中未遇到的场景的原因之一。在下文中，我们讨论了研究人员
开始将物理知识纳入学习过程的一些方法，以便 ML 模型能够捕捉符合既定物理定律的可
概括动态模式。

Physics-based loss to improve predictions One of the most common techniques to make ML
models consistent with physical laws is to incorporate physical constraints into the loss function
of ML models as follows [Karpatne et al., 2017a],

基于物理的损耗改善预测使 ML 模型符合物理定律的最常见技术之一是将物理约束纳入 ML
模型的损耗函数中，如下所示[Karpatne 等人，2017a]，

where the training loss LossTRN measures a supervised error (e.g., RMSE or cross-entropy)
between true labels Ytrue and predicted labels Ypred, and λ is a hyper-parameter to control the
weight of model complexity loss R(W ). These first two terms are the standard loss of ML models.
The addition of physics-based loss LossPHY aims to ensure consistency with physical laws. An
added benefit is that training can include unlabeled data by omitting LossTRN.
其中，训练损失度量真实标签 Ytrue 和预测标签 Ypred 之间的监督误差（例如，RMSE 或交
叉熵），λ 是控制模型复杂度损失 R（W）权重的超参数。前两项是 ML 模型的标准损失。
添 加基 于物 理的 损耗 损耗 旨在 确保 与物 理定 律的 一致 性。 另一 个好 处是 ，通 过省 略
LossTRN，训练可以包括未标记的数据。

In the context of complex natural systems, physics-based losses have shown great success in
improving prediction ability. A number of researchers [Karpatne et al., 2017b; Muralidhar et al.,
2018] have studied how to incorporate monotonic physical relationships as constraints in loss
functions for a neural network (NN). In the context of lake temperature modeling, [Karpatne et
al., 2017b] includes an additional physics-based penalty that ensures denser water predictions
remain at lower depths, a known monotonic relationship. [Jia et al., 2019; Read et al., 2019]
further extended this work to capture even more complex and general physical relationships that
happen on a temporal scale. Specifically, they use a physics-based penalty for energy
conservation in the loss function to ensure the lake thermal energy gain across time is consistent
with the net thermodynamic fluxes going in and out of the lake. Other work in parameterization
[Beucler et al., 2019] has also shown it is possible to enforce energy conservation laws in
emulating cloud processes.

在复杂自然系统的背景下，基于物理的损耗在提高预测能力方面取得了巨大成功。许多研
究人员[Karpatne 等人，2017b；Muralidhar 等人，2018]研究了如何将单调物理关系作为约
束纳入神经网络（NN）的损失函数中。在湖温建模的背景下，[Karpatne 等人，2017b]包括
一个额外的基于物理的惩罚，以确保在较低深度保持更密集的水预测，这是一个已知的单
调关系。[Jia 等人，2019；Read 等人，2019]进一步扩展了这项工作，以捕捉在时间尺度上
发生的更复杂和一般的物理关系。具体来说，他们在损失函数中使用基于物理的能量守恒
惩罚，以确保湖泊热能随时间的增加与进出湖泊的净热力学通量一致。参数化方面的其他
工作[Beucler 等人，2019]也表明，在模拟云过程中执行能量守恒定律是可能的。

Another strand of work incorporating loss function alterations is in solving PDEs, in which
adherence to the governing equations is enforced in the loss function. In [Raissi et al., 2019a],
this concept is developed and shown to create data-efficient spatiotemporal function
approximators to both solve and find parameters of basic PDEs like Burgers Equation or the
Schrodinger Equation. In [Raissi et al., 2019b], this is extended to cover inverse modeling
problems for quantities of interest in vortex induced vibrations. Beyond a simple feed-forward
network, [Zhu et al., 2019] proposes an encoder-decoder for predicting transient PDEs with
governing PDE constraints, which was further extended to deep auto-regressive dense encoder-
decoders with a Bayesian framework using stochastic weight averaging to quantify uncertainty in
[Geneva and Zabaras, 2020].

另一项包含损失函数更改的工作是求解偏微分方程，其中在损失函数中强制遵守控制方程
在[Raissi 等人，2019a]中，开发并证明了该概念，以创建数据高效的时空函数近似器，以
求 解和 查找 基本 偏微 分方 程（ 如 Burgers 方 程或 薛定 谔方 程） 的参 数。 在 [Raissi 等 人 ，
2019b]中，这被扩展到涵盖涡激振动中感兴趣量的反建模问题。除了简单的前馈网络之外 ，
[Zhu 等人，2019]提出了一种编码器-解码器，用于预测具有控制偏微分方程约束的瞬态偏
微分方程，并在[Geneva 和 Zabaras，2020]中进一步扩展到具有贝叶斯框架的深度自回归密
集编码器-解码器，使用随机权重平均量化不确定性。

Certain aspects of dynamical systems modeling have also shown promise in informing loss
functions for improved prediction beyond that of the physics model. In [Erichson et al., 2019],
they penalize autoencoders based on physicallymeaningful stability measures in dynamical
systems for improved prediction of fluid flow and sea surface temperature.

动态系统建模的某些方面也显示出在通知损失函数方面的前景，以改进超越物理模型的预
测。在[Erichson 等人，2019]中，他们基于动力系统中具有物理意义的稳定性度量来惩罚自
动编码器，以改进对流体流动和海面温度的预测。

They showed an improved mapping of past states to future states in both modeling scenarios in
addition to improved generalizability to new data.

他们展示了在两种建模场景中改进的过去状态到未来状态的映射，以及改进的新数据泛化
能力。

Though these loss functions are mostly seen in common variants of neural networks, they can
also be seen in architectures like echo state networks [Doan et al., 2019]. In this work they find
integrating the physics-based loss from the governing equations in a Lorenz system, a commonly
studied system in dynamical systems, strongly improves the echo state network’s time-accurate
prediction of the system and also reduces convergence time.

虽然这些损失函数大多出现在神经网络的常见变体中，但它们也可以出现在回声状态网络
等架构中[Doan 等人，2019]。在这项工作中，他们发现，将洛伦兹系统（动力学系统中的
一个常用研究系统）中基于物理的损耗与控制方程相结合，大大提高了回声状态网络对系
统的时间精确预测，并缩短了收敛时间。

Auxiliary Task
in Multi-Task Learning Multi-task learning frameworks allow for multiple learning tasks to be
solved at the same time, ideally while exploiting commonalities and differences across tasks. This
can result in improved learning efficiency and predictions for one or more of the tasks.

多任务学习
多任务学习框架中的辅助任务允许同时解决多个学习任务，理想情况下，同时利用任务之
间的共性和差异。这可以提高一个或多个任务的学习效率和预测。

Therefore, another way to implement physics-based learning constraints is to use an auxiliary

task in a multi-task learning framework. This is shown to be successful in computer vision [Zhang
et al., 2014], but a physics-based analogue would be to have an auxiliary task representing
consistency with physics-based principles. In this paradigm, a task-constrained loss function can
be formulated to allow errors of related tasks to be back-propagated jointly to improve model
generalization. Early work in computational chemistry showed that a NN could be trained to
predict energy by constructing a loss function that had penalties for both inaccuracy and
inaccurate energy derivatives with respect to time as determined by the surrounding energy
force field [Pukrittayakamee et al., 2009].

因此，实现基于物理的学习约束的另一种方法是在多任务学习框架中使用辅助任务。这在
计算机视觉中被证明是成功的[Zhang 等人，2014]，但基于物理的模拟将具有表示与基于物
理的原理一致性的辅助任务。在此范式中，可以制定任务约束损失函数，以允许相关任务
的错误联合反向传播，以提高模型泛化。计算化学的早期工作表明，可以通过构造损失函
数来训练神经网络以预测能量，该损失函数对周围能量力场确定的时间不准确和不准确的
能量导数都有惩罚[Pukritayakamee 等人，2009]。

Then in particle physics, [de Oliveira et al., 2017] uses an additional task for the discriminator
network in a general adversarial network (GAN) to satisfy certain properties of particle
interaction for the production of jet images of particle energy.

然后，在粒子物理学中，[de Oliveira 等人，2017]在一般对抗网络（GAN）中使用了一个额

外的鉴别器网络任务，以满足粒子相互作用的某些特性，以产生粒子能量的射流图像。

Physics-Guided Initialization Since many ML models require an initial choice of model parameters
before training, researchers have explored different ways to physically inform a model starting
state. Poor initialization can cause models to anchor in local minima, which is especially true for
deep neural networks. Transfer learning can effectively tackle this issue, where the pre-trained
models from a related task are fine-tuned with limited training data to fit the desired task.

物理引导初始化由于许多 ML 模型在训练之前需要初始选择模型参数，研究人员探索了物
理通知模型启动状态的不同方法。初始化不良会导致模型锚定在局部极小值，这对于深度
神经网络尤其如此。转移学习可以有效地解决这个问题，即使用有限的训练数据对相关任
务的预训练模型进行微调，以适合所需的任务。

One way to harness physics-based modeling knowledge is to use the physics-based model’s
simulated data to pre-train the ML model, which also alleviates data paucity issues.

利用基于物理的建模知识的一种方法是使用基于物理的模型的模拟数据预训练 ML 模型，
这也缓解了数据不足的问题。

Jia et al. extensively discusses this strategy [Jia et al., 2019]. They pre-train their Physics-Guided
Recurrent Neural Network (PGRNN) models for lake temperature modeling on simulated data
generated from a physics-based model and fine tune the NN with little observed data. They show
that pre-training, even using data from a physical model with an incorrect set of parameters, can
still significantly reduce the training data needed for a quality model. In addition, [Read et al.,
2019] show that such models are able to generalize better to unseen scenarios than pure physics-
based models.

贾等人广泛讨论了这一战略[Jia 等人，2019]。他们根据基于物理的模型生成的模拟数据预
先训练其物理引导的递归神经网络（PGRNN）模型，用于湖泊温度建模，并使用少量观测
数据微调神经网络。他们表明，预训练，即使使用来自具有错误参数集的物理模型的数据
仍然可以显著减少质量模型所需的训练数据。此外，[Read 等人，2019]表明，与纯物理模
型相比，此类模型能够更好地概括未知场景。

3.2 Physics-Guided Design of Architecture

3.2 物理引导的建筑设计

Although the physics-based loss in the previous section helps constrain the search space of
neural networks during training, the neural network architecture is often still a black box.

虽然前一节中基于物理的损耗有助于在训练期间约束神经网络的搜索空间，但神经网络架
构通常仍然是一个黑匣子。

There are usually no architectural properties to implicitly encode physical consistency or other
desired physical properties. A recent research direction has been to construct new ML
architectures for the tasks defined in Section 2 with this in mind.

通常没有架构属性来隐式编码物理一致性或其他所需的物理属性。考虑到这一点，最近的
一个研究方向是为第 2 节中定义的任务构建新的 ML 架构。

Intermediate Physical Variables One way to embed well-known physical principles into NN design
is to ascribe physical meaning for certain neurons in the NN by computing physically relevant
intermediate variables in the neural pathway from inputs to outputs. In lake temperature
modeling, [Daw et al., 2019] incorporates a physical intermediate variable in the LSTM
architecture as shown in Figure 1. This model produces physically consistent predictions in
addition to appending a dropout layer to quantify uncertainty. In [Muralidhar et al., 2018], a
similar approach is taken to insert physics-constrained variables as the intermediate variables in
the convolutional neural network (CNN) architecture and achieve significant improvement over
state-of-the-art physics-based models on the problem of predicting drag force on particle
suspensions in moving fluids.

中间物理变量将众所周知的物理原理嵌入神经网络设计的一种方法是通过计算神经通路中
从输入到输出的物理相关中间变量来赋予神经网络中某些神经元的物理意义。在湖温建模
中，[Daw 等人，2019]将物理中间变量纳入 LSTM 架构中，如图 1 所示。该模型除了附加一
个退出层来量化不确定性之外，还产生了物理上一致的预测。在 [Muralidhar 等人，2018
年]中，采取了类似的方法，将物理约束变量作为中间变量插入卷积神经网络（CNN）架构
中，并在预测运动流体中颗粒悬浮物上的阻力问题上，显著改善了基于最先进物理的模型。
Figure 1: Monotonicity-preserving LSTM Architecture. Components in red represent novel
physics-informed innovations in LSTM [Daw et al., 2019]

图 1 ：保 持单 调性 的 LSTM 架 构。 红色 组件 表示 LSTM 中新 的物 理信 息创 新 [Daw 等 人 ，

2019]

An additional benefit of adding physically relevant intermediate variables in an ML architecture is

that they can help extract physically meaningful hidden representation that can be interpreted by
domain scientists. This is particularly valuable, as standard deep learning models are limited in
their interpretability since they can only extract abstract hidden variables using highly complex
connected structure. This is further exacerbated given the randomness involved in the
optimization process.

在 ML 架构中添加物理相关中间变量的另一个好处是，它们可以帮助提取物理意义的隐藏
表示，可以由领域科学家解释。这尤其有价值，因为标准深度学习模型的可解释性有限，
因为它们只能使用高度复杂的连接结构提取抽象的隐藏变量。考虑到优化过程中涉及的随
机性，这一情况进一步加剧。

Encoding invariances Common neural network design paradigms like the RNN and CNN have
revolutionized the ability of ML algorithms by implicitly encoding time invariance into the RNN
architecture and spatial translation, rotation, and scale invariance into the CNN. In the same way,
scientific modeling tasks may require other invariances based on physical laws. In turbulence
modeling and fluid dynamics, recent work defines a tensor basis neural network to embed the
fundamental principle of rotational invariance into a NN [Ling et al., 2016] for improved
prediction accuracy. This solves a key problem in ML models for turbulence modeling because
without rotational invariance, the model evaluated on identical flows with axes defined in other
directions could yield different predictions.

编码不变性常见的神经网络设计范式，如 RNN 和 CNN，通过将时间不变性隐式编码到 RNN

架构中，并将空间平移、旋转和尺度不变性隐式编码到 CNN 中，彻底改变了 ML 算法的能
力。同样，科学建模任务可能需要基于物理定律的其他不变性。在湍流建模和流体动力学
中，最近的工作定义了张量基神经网络，将旋转不变性的基本原理嵌入到神经网络中[Ling
等人，2016]，以提高预测精度。这解决了湍流建模 ML 模型中的一个关键问题，因为没有
旋转不变性，在其他方向定义轴的相同流上评估的模型可能会产生不同的预测。

This work alters the NN architecture by adding a higher-order multiplicative layer that ensures
the prediction lies on a rotationally invariant tensor basis.
这项工作通过添加一个高阶乘法层来改变神经网络的结构，该层确保预测位于旋转不变张
量的基础上。

Architectural modifications in the context of solving PDEs Architecture modifications are also
seen extensively in dynamical systems research involving differential equations. In [Chen et al.,
2018a], a continuous depth neural network based on the Residual Network [He et al., 2016], is
proposed for solving ordinary differential equations. They change the traditionally discretized
neuron layer depths into continuous equivalents such that hidden states can be parameterized by
differential equations in continuous time. Then, in pioneering work by [Ruthotto and Haber,
2018], three variations of CNNs are proposed for solving PDEs. Each variation uses mathematical
theories to guide the design of the CNN based on fundamental properties of the PDEs. Multiple
types of modifications are made, including adding symmetry layers to guarantee stability
expressed by the PDEs and layers that convert inputs to kinematic eigenvalues that satisfy certain
physical properties. They define a parabolic CNN inspired by anisotropic filtering, a hyperbolic
CNN based on Hamiltonian systems, and a second order hyperbolic CNN. Hyperbolic CNNs were
found to preserve the energy in the system as intended, which set them apart from parabolic
CNNs that smooth the output data, reducing the energy. In a similar application, [Chang et al.,
2019] uses principles from the stability properties of differential equations in dynamical systems
modeling to guide the design of gating mechanism and activation functions in an RNN.

在求解偏微分方程时的结构修改在涉及微分方程的动力系统研究中也广泛存在。 [Chen 等
人，2018a]提出了一种基于残差网络的连续深度神经网络[He 等人，2016]，用于求解常微
分方程。他们将传统离散化的神经元层深度更改为连续等价物，从而可以通过连续时间的
微分方程参数化隐藏状态。然后，在[Ruthotto 和 Haber，2018]的开创性工作中，提出了三
种 CNN 变体来求解偏微分方程。每种变体都使用数学理论来指导基于偏微分方程基本特性
的 CNN 设计。进行了多种类型的修改，包括添加对称层以保证由偏微分方程表示的稳定性 ，
以及将输入转换为满足某些物理特性的运动学特征值的层。他们定义了受各向异性滤波启
发的抛物线 CNN、基于哈密顿系统的双曲 CNN 和二阶双曲 CNN。发现双曲 CNN 可以按预期
保存系统中的能量，这使其与抛物线 CNN 不同，抛物线 CNN 可以平滑输出数据，减少能量。
在类似的应用中，[Chang 等人，2019]使用动态系统建模中微分方程稳定性的原理来指导
RNN 中选通机制和激活函数的设计。

Physics-Guided Neural Architecture Search Currently employed architectures primarily have been
developed manually by human experts, which can be a timeconsuming and error-prone process.
Because of this, there is growing interest in automated neural architecture search methods
[Elsken et al., 2018]. Though outside of the objectives defined in Section 2, a young but promising
direction in ML architecture design is to embed prior physical knowledge into neural architecture
searches. In [Ba et al., 2019a], physically meaningful input nodes and physical operations
between nodes are added to the neural architecture search space for the search algorithm to
discover more ideal physics-guided machine learning architectures.

物理引导的神经架构搜索目前使用的架构主要是由人类专家手动开发的，这可能是一个耗
时且容易出错的过程。因此，人们对自动神经架构搜索方法越来越感兴趣 [Elsken 等人，
2018]。尽管在第 2 节定义的目标之外，ML 架构设计中一个年轻但有希望的方向是将先验
物理知识嵌入到神经架构搜索中。在[Ba 等人，2019a]中，将具有物理意义的输入节点和节
点之间的物理操作添加到神经架构搜索空间中，用于搜索算法，以发现更理想的物理引导
机器学习架构。

3.3 Residual modeling

3.3 残余建模

The oldest and most common approach for directly addressing the imperfection of physics-based
models in the scientific community is residual modeling, where an ML model (usually linear
regression) learns to predict the errors, or residuals, made by a physics-based model [Forssell and
Lindskog, 1997]. The key concept is to learn biases of the physical model (relative to
observations) and use it to make corrections to the physical model’s predictions. As an example,
in [Kani and Elsheikh, 2017] a physics-driven ”deep residual recurrent neural network (DR-RNN)”
is proposed to find the residual minimiser of numerically discretized PDEs. Their architecture
involves a stacked RNN embedded with the dynamical structure of the PDEs such that each layer
of the NN solves a layer of the residual equations. They showed DRRNN to sharply reduce both
computational cost and time discretization error of the reduced order modeling framework.

在科学界，直接解决基于物理的模型缺陷的最古老和最常见的方法是残差建模，其中 ML
模 型 （ 通 常 是 线 性 回 归 ） 学 习 预 测 基 于 物 理 的 模 型 产 生 的 误 差 或 残 差 [Forssell 和
Lindskog，1997]。关键概念是学习物理模型的偏差（相对于观察值），并使用它对物理模
型的预测进行修正。例如，[Kani 和 Elsheikh，2017]提出了一种物理驱动的“深度残差递归
神经网络（DR-RNN）”来寻找数值离散偏微分方程的残差最小值。他们的架构包括嵌入偏
微分方程动态结构的堆叠神经网络，使得神经网络的每一层求解一层残差方程。他们表明
DRRNN 可以显著降低降阶建模框架的计算成本和时间离散化误差。

Recently, researchers have begun to use ML models more sophisticated than regression [Wan et
al., 2018] for residual modeling. However, one key limitation in residual modeling is the physics-
based constraints (like in Section 3.1) are hard to enforce because such approaches model the
errors instead of the physical quantities physics-based problems.

最近，研究人员开始使用比回归更复杂的 ML 模型进行残差建模[Wan 等人，2018]。然而，

残差建模的一个关键限制是基于物理的约束（如第 3.1 节）难以实施，因为此类方法建模
的是误差，而不是基于物理量的问题。

Figure 2: Machine Learning model fM L trained to model the error made by Physics model fP HY
图 2：机器学习模型 fM L 训练用于建模物理模型 fP HY 产生的误差。

More recently, a key area in which residual modeling has been applied is in reduced order models
(ROMs) of dynamical systems. ROMs of dynamical systems are simplified versions of high fidelity,
PDE-based numerical simulations that are much cheaper computationally. Usually, this involves
techniques like principal components analysis to project governing equations onto a low-
dimensional subspace spanned by basis functions [Kerschen et al., 2005]. After reducing model
complexity, an ML model can be used to model the residual due to the truncation. ROM methods
were created in response to the problem of many detailed simulations being too expensive to be
used in various engineering tasks including design optimization and real-time decision support. In
[San and Maulik, 2018], a simple NN used to model the error due to the model reduction is
shown to sharply reduce high error regions when applied to Bousinessq equations. Also, in [Wan
et al., 2018], an RNN is used to model the residual between a ROM for prediction of extreme
weather events and the available data projected to a reduced-order space.

最近，残差建模的一个关键领域是动态系统的降阶模型。动力系统的 ROM 是高保真、基于

偏微分方程的数值模拟的简化版本，计算成本更低。通常，这涉及到主成分分析等技术，
以将控制方程投影到由基函数跨越的低维子空间上[Kerschen 等人，2005]。在降低模型复杂
度后，可以使用 ML 模型对截断引起的残差进行建模。ROM 方法是为了应对许多详细模拟
过于昂贵而无法用于各种工程任务（包括设计优化和实时决策支持）的问题而创建的。在
[San 和 Maulik，2018]中，当应用于 Bousinessq 方程时，用于建模由于模型缩减而产生的误
差的简单神经网络可以显著减少高误差区域。此外，在[Wan 等人，2018]中，RNN 用于建
模极端天气事件预测 ROM 和投影到降阶空间的可用数据之间的残差。

3.4 Hybrid Physics-ML Models

3.4 混合物理 ML 模型

Residual modeling is just one way to combine physics-based models with ML where both are
operating simultaneously.

残差建模只是将基于物理的模型与 ML 结合起来的一种方法，两者同时运行。

We will call a generalized version of this Hybrid Physics-ML Models.

我们将这种混合物理 ML 模型的广义版本称为 ML 模型。

One straightforward method to combine physics-based and ML models is to feed the output of a
physics-based model as input to an ML model. [Karpatne et al., 2017b] showed that using the
output of a physics-based model as one feature in an ML model along with inputs used to drive
the physicsbased model for lake temperature modeling can improve predictions. Visualization of
this method is shown in Figure 3.

将基于物理的模型和 ML 模型相结合的一种简单方法是将基于物理的模型的输出作为 ML 模
型的输入。[Karpatne 等人，2017b]表明，将基于物理的模型的输出作为 ML 模型中的一个
特征，以及用于驱动基于物理的模型进行湖泊温度建模的输入，可以改进预测。该方法的
可视化如图 3 所示。

Figure 3: Diagram of a hybrid physics-ML model [Karpatne et al., 2017b]

图 3：混合物理 ML 模型图[Karpatne 等人，2017b]

Another variant of hybrid physics-data models is to use an ML model to predict intermediate

quantities in a physicsbased model. Recent work in fluid dynamics [Parish and Duraisamy, 2016]
proposes a NN to estimate variables in the closure model to account for missing physics to
improve predictions of discrepency of RANS solvers. They show this correction to traditional
turbulence models results in convincing improvements of predictions. In [Hamilton et al., 2017], a
subset of the mechanistic model’s equations are replaced with data-driven nonparametric
methods to improve prediction beyond the baseline process model. Then, in [Zhang et al., 2018],
a physics-based solving architecture for power system state estimation embeds a deep learning
model in place of traditional predicting and optimization techniques.

混合物理数据模型的另一个变体是使用 ML 模型预测基于物理的模型中的中间量。流体动
力学方面的最新工作[Parish 和 Duraisamy，2016]提出了一种神经网络来估计闭合模型中的
变量，以解释缺失的物理现象，从而改进 RANS 解算器离散性的预测。他们表明，对传统
湍流模型的这种修正使预测得到了令人信服的改进。在[Hamilton 等人，2017]中，用数据
驱动的非参数方法取代了机械模型方程的子集，以改善基线过程模型之外的预测。然后，
在[Zhang 等人，2018]中，基于物理的电力系统状态估计求解架构嵌入了深度学习模型，取
代了传统的预测和优化技术。

Hybrid models for parameterization take a similar form, where complex physical processes are
too difficult or expensive for numerical process models to simulate and an ML model can be
substituted. In geodynamics modeling, [Goldstein et al., 2014] showed that an ML model for
oscillatory flow ripples can be substituted into an established model for geologic material being
moved by fluid flow. They showed that the hybrid model was able to capture dynamics previously
absent from the model, which led to the discovery of two new observed pattern modes in the
geology.

用于参数化的混合模型采用类似的形式，其中复杂的物理过程对于数值过程模型来说过于
困难或昂贵，可以用 ML 模型代替。在地球动力学建模中，[Goldstein 等人，2014]表明，振
荡流涟漪的 ML 模型可以替换为流体移动地质物质的既定模型。他们表明，混合模型能够
捕捉到以前模型中没有的动力学，这导致在地质学中发现了两种新的观察模式。
In another class of hybrid frameworks, the overall prediction is a weighted sum of predictions
from a physical model and an ML model, where the weights depend on prediction circumstance.
For example, oftentimes in physics-based models long-range interactions are more easily
modeled by existing physics equations (e.g. gravity) than more stochastic short-range interactions
(e.g. quantum mechanics). Hybrid frameworks have been used to adaptively combine ML
predictions for short-range processes and physics model predictions for long-range processes for
applications in chemical reactivity [Yao et al., 2018] and seismic activity prediction [Paolucci et al.,
2018]. Estimator quality at a given time and location can also be used to determine whether a
prediction comes from the physical model or the ML model, which was shown in [Chen et al.,
2018b] for air pollution estimation and in [Vlachas et al., 2018] for dynamical system forecasting
more generally.

在另一类混合框架中，整体预测是来自物理模型和 ML 模型的预测的加权和，其中权重取
决于预测环境。例如，在基于物理的模型中，与更随机的短程相互作用（例如量子力学）
相比，长程相互作用通常更容易由现有的物理方程（例如重力）建模。混合框架已被用于
自适应地结合短程过程的 ML 预测和长程过程的物理模型预测，以应用于化学反应性 [Yao
等人，2018]和地震活动预测[Paolucci 等人，2018]。给定时间和位置的估计器质量也可用于
确定预测是来自物理模型还是 ML 模型，这在[Chen 等人，2018b]中用于空气污染估计，在
[Vlachas 等人，2018]中用于更一般的动力系统预测。

Moreover, in inverse modeling, there is a growing use of hybrid models that first use physics-
based models to perform the direct inversion, then use deep learning models to further refine
the inverse model’s predictions. Multiple works have shown an effective application for this in
computed tomography (CT) reconstructions [Jin et al., 2017; Bubba et al., 2019]. Another
common technique in inverse modeling of images (e.g. medical imaging, particle physics
imaging), is the use of CNNs as deep image priors [Ulyanov et al., 2018]. In [Senouf et al., 2019],
they embed both a CNN that serves as the image prior for a physics-based forward model for
MRIs to exploit data and prior knowledge simultaneously.

此外，在逆建模中，混合模型的使用越来越多，首先使用基于物理的模型进行直接反演，
然后使用深度学习模型进一步细化逆模型的预测。多项研究表明，这一点在计算机断层扫
描（CT）重建中得到了有效应用[Jin 等人，2017 年；Bubba 等人，2019 年]。图像逆建模
（例如医学成像、粒子物理成像）中的另一种常见技术是使用 CNN 作为深度图像先验
[Ulyanov 等人，2018]。在[Senouf 等人，2019]中，他们嵌入了一个 CNN，作为磁共振成像
基于物理的正向模型的图像先验，以同时利用数据和先验知识。

4 Discussion, Future Directions and Concluding Remarks

4 讨论、未来方向和结论

Given the current deluge of sensor data and advances in ML methods amidst pressing
environmental and physical modeling problems facing scientists, we envision the merging of
principles from ML and physics to play an invaluable role in the future of scientific modeling.
Although both physicsbased modeling and ML are rapidly advancing fields, knowledge integration
for mutual gain requires clear lines of communication between disciplines. Currently, most of the
diverse set of techniques for bringing ML and physical knowledge together have been developed
in isolated applications by researchers working in distinct disciplines.

考虑到当前传感器数据的泛滥以及 ML 方法在科学家面临的紧迫环境和物理建模问题中的
进步，我们设想将 ML 原理与物理学相结合，在未来的科学建模中发挥宝贵作用。尽管基
于物理的建模和 ML 都是快速发展的领域，但为了实现互利共赢，知识集成需要学科之间
的明确沟通。目前，将 ML 和物理知识结合在一起的大多数不同技术都是由不同学科的研
究人员在独立应用中开发的。

In this survey, we reviewed a number of techniques for integrating physics-based knowledge with
machine learning. We classified these methodologies into four categories; Physics-Guided
Learning, Physics-Guided Design of Architecture, Residual Modeling, and Hybrid Physics-ML
Modeling. From the early results reported in some of the papers, we can see that the
incorporation of prior physics-based knowledge can alleviate some of the most significant
problems with black box ML approaches: the requirement of large datasets, inconsistency with
physical laws, and the struggle to extrapolate to unseen scenarios [Read et al., 2019]. This
increase in data efficiency and physical consistency serves to benefit several application specific
research objectives including inverse modeling, parameterization, discovery of governing
equations, and solving PDEs.

在本次调查中，我们回顾了一些将基于物理的知识与机器学习相结合的技术。我们将这些
方法分为四类；物理引导学习、物理引导架构设计、残差建模和混合物理 ML 建模。从一
些论文中报告的早期结果中，我们可以看出，结合先前基于物理的知识可以缓解黑箱 ML
方法的一些最重要问题：对大型数据集的要求、与物理定律的不一致性，以及难以外推到
看不见的场景[Read 等人，2019]。数据效率和物理一致性的提高有助于实现几个特定于应
用的研究目标，包括逆建模、参数化、发现控制方程和求解偏微分方程。

From Table 1, it is easy to see that a large amount of research has been done in the context of
developing integrated physics and ML models that are able to improve predictions beyond that
can be obtained using the state of the art physical models (row 1 in Table 1). Also note that many
boxes in the Table 1 are empty, many of which represent opportunities for future work. For
example, combining ML and physics models for better parameterizations is increasingly being
used in domains like climate science and weather forecasting [Krasnopolsky and Fox-Rabinovitz,
2006], but most of these works have relied on simple ML models. Furthermore, principles from
super-resolution frameworks are beginning to be applied to downscaling to create higher
resolution climate predictions [Vandal et al., 2017]. All of these approaches can benefit from
more powerful approaches listed here (e.g., physics based loss, intermediate physics variables,
etc).

从表 1 很容易看出，在开发集成物理和 ML 模型的背景下进行了大量研究，这些模型能够
改进使用最先进的物理模型（表 1 第 1 行）可以获得的预测。还要注意，表 1 中的许多框
是空的，其中许多框表示未来工作的机会。例如，将 ML 和物理模型相结合以实现更好的
参 数 化 正 越 来 越 多 地 用 于 气 候 科 学 和 天 气 预 报 等 领 域 [Krasnopolsky 和 Fox
Rabinovitz，2006]，但这些工作大多依赖于简单的 ML 模型。此外，超分辨率框架的原则开
始应用于缩小尺度，以创建更高分辨率的气候预测 [Vandal 等人，2017]。所有这些方法都
可以受益于此处列出的更强大的方法（例如，基于物理的损耗、中间物理变量等）。

Two emerging research directions in Physics-ML integration not previously mentioned in this
survey include efforts in uncertainty quantification (UQ) and data generation. Stateof-the-art
physics-based models for simulating real systems are often characterized by vast numbers of
input parameters, and quantifying uncertainty of these models using traditional methods like
Monte Carlo is usually infeasible due to the thousands or millions of forward model evaluations
needed to obtain convergent statistics. ML models, especially when augmented with physics-
based knowledge, can serve as surrogate models for faster evaluation of uncertainty [Yang and
Perdikaris, 2019a]. For data generation, generative ML models can be used in a manner similar to
simulations by physicsbased models. Using generative models as opposed to physical simulation
often incurs certain benefits, including reduced computation time and being able to represent
complex data spaces without knowing the entire topology. Furthermore, generative models are
often used as representations of solution spaces to complex physical and dynamical systems.

本调查中未提及的物理学 ML 集成的两个新兴研究方向包括不确定性量化（UQ）和数据生
成。用于模拟真实系统的最先进的基于物理的模型通常具有大量输入参数的特点，并且由
于需要数千或数百万个正向模型评估来获得收敛统计信息，因此使用传统方法（如蒙特卡
洛）来量化这些模型的不确定性通常是不可行的。ML 模型，尤其是当添加基于物理的知识
时，可以作为替代模型，用于更快地评估不确定性[Yang 和 Perdikaris，2019a]。对于数据生
成，生成 ML 模型的使用方式类似于基于物理的模型的模拟。与物理模拟相反，使用生成
模型通常会带来某些好处，包括减少计算时间，并且能够在不知道整个拓扑的情况下表示
复杂的数据空间。此外，生成模型通常用于表示复杂物理和动力学系统的解空间。

Many of the approaches discussed in this work can apply to generative models as well [Shah et
al., 2019]. More works in uncertainty quantification and data generation can be found in Table 1.

本研究中讨论的许多方法也适用于生成模型[Shah 等人，2019]。不确定性量化和数据生成
方面的更多工作见表 1。

Acknowledgements This work was supported by NSF grant #1934600 and by DARPA award
W911NF-18-1-0027.

致谢：这项工作得到了 NSF 拨款#1934600 和 DARPA 奖 W911NF-18-1-0027 的支持。