Ubc 2022 May Gupta Rachit

Deep Learning-based Reduced Order Modeling for
Unsteady Flow Dynamics and Fluid-Structure Interaction
by
Rachit Gupta
B.Tech., Delhi Technological University, 2019
A THESIS SUBMITTED IN PARTIAL FULFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OF
Master of Applied Science
in
THE FACULTY OF GRADUATE AND POSTDOCTORAL

STUDIES
(Mechanical Engineering)
The University of British Columbia

(Vancouver)
January 2022
© Rachit Gupta, 2022

The following individuals certify that they have read, and recommend to the Fac-
ulty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled:
Deep Learning-based Reduced Order Modeling for Unsteady Flow Dy-

namics and Fluid-Structure Interaction
submitted by Rachit Gupta in partial fulfillment of the requirements for the degree
of Master of Applied Science in Mechanical Engineering.
Examining Committee:
Rajeev Jaiman, Associate Professor, Mechanical Engineering, UBC
Supervisor
Ryozo Nagamune, Professor, Mechanical Engineering, UBC
Supervisory Committee Member
Mauricio Ponga, Assistant Professor, Mechanical Engineering, UBC
Supervisory Committee Member
ii
Abstract
This work presents data-driven predictions of nonlinear dynamical systems involv-

ing unsteady flow and fluid-structure interaction. Of particular interest is to develop
a new simulation framework integrating high-fidelity models with deep learning to-
wards Digital Twin. The final goal is to learn and predict the coupled dynamics via
the digital twin of ship vessels and propellers. End-to-end deep learning-based
reduced order models (DL-ROMs) are presented for digital twin development.
The first part of this study develops an overall framework for DL-ROMs. The
emphasis is to investigate the predictive performance of the hybrid DL-ROMs,
which vary in obtaining the low-dimensional features, i.e., proper orthogonal de-
composition (POD) and convolutional autoencoders. The low-dimensional features
are evolved in time using recurrent neural networks (RNNs). This leads to the
formulation of two DL-ROM frameworks: the POD-RNN and the convolutional
recurrent autoencoder network (CRAN). To assess data-driven predictions, POD-
RNN and CRAN are applied to predict unsteady flows and instantaneous forces
for flow past static bluff bodies. We perform flow prediction analysis for a config-
uration of side-by-side cylinders with wake interference. For systems with moving
interfaces and three-dimensional (3D) geometries, we develop modular DL-ROM
techniques.
The second part of this study includes model reduction strategies to predict
vortex-induced vibration and 3D unsteady flows. The knowledge gained in the
previous parts is utilized to develop partitioned and scalable DL-ROMs for un-
steady flows with moving interfaces and parametric effects. We first develop a
partitioned DL-ROM framework for fluid-structure interaction. The novel multi-
level DL-ROM combines the effect of POD-RNN and CRAN by modular learning
iii
of two physical fields independently. While POD-RNN provides extraction of the
fluid-structure interface, the CRAN enables the prediction of flow fields. For time
series prediction of 3D flows, we present a 3D CRAN-based framework for pre-
dicting the fluid forces and vortex shedding patterns. We provide an assessment of
improving learning capabilities using transfer learning for complex 3D flows with
variable Reynolds numbers. The simplicity and computational efficiency of the
proposed DL-ROMs allow investigation for various geometries and physical pa-
rameters. This research opens ways for digital twin development for near real-time
prediction of unsteady flows and fluid-structure interaction.
iv
Lay Summary
Large-scale simulations of fluid-structure systems have been possible using high-

performance computing (HPC). These high-fidelity simulations using coupled non-
linear partial differential equations are useful for marine/offshore engineering ap-
plications. Despite efficient numerical computing, state-of-the-art computational
fluid dynamics (CFD) simulations are quite inefficient for real-time predictions.
This study presents advances in integrating HPC-based computations with data
science and machine learning. The primary focus is to: (i) explore the integra-
tion of model reduction strategies with deep learning and (ii) develop modular and
scalable data-driven frameworks for unsteady flow and fluid-structure analysis. A
series of test cases are studied to investigate the integration of standard CFD codes
with model reduction and deep learning techniques to predict vortex-induced mo-
tion and unsteady flows. Parametric predictions of three-dimensional wake flows
are also explored. Such hybrid CFD with deep learning is aligned with the cur-
rent marine/offshore industry needs for real-time predictions and structural health
monitoring via digital twin.
v
Preface
This M.A.Sc. thesis entitled “Deep Learning-based Reduced Order Modeling

for Unsteady Flow Dynamics and Fluid-Structure Interaction” presents the
original research conducted by Rachit Gupta under the supervision of Dr. Rajeev
Jaiman.
Chapters 1, 2, 3, 5, 6 and 7: The presented introductory in Chapter 1 (Section
1.1), literature review in Chapter 2, methodologies in Chapter 3, results and dis-
cussions in Chapters 5 and 6, and parts of the conclusions and recommendations
in Chapter 7 (Sections 7.1 and 7.2) have been published in the following journal
articles:
R. Gupta and R. Jaiman. A hybrid partitioned deep learning methodology
for moving interface and fluid–structure interaction. Computers & Fluids, 233:
105239, 2022.
R. Gupta and R. Jaiman. Three-dimensional deep learning-based reduced order
model for unsteady flow dynamics with variable reynolds number. arXiv preprint
arXiv:2112.09302, 2021.
The credit authorship contribution corresponding to authors are as follows:
Rachit Gupta: Writing - original draft, Coding and Debugging, Investigation, Post-
processing, Data analysis. Rajeev Jaiman: Conceptualization, Methodology, Anal-
ysis, Editing, Supervision.
Chapters 4 and 7: Results and discussions presented in Chapter 4 and parts of
conclusions in Chapter 7 (Section 7.1) are published in the following co-authored
paper:
S. R. Bukka, R. Gupta, A. R. Magee, and R. K. Jaiman. Assessment of unsteady
flow predictions using hybrid deep learning based reduced-order models. Physics
vi
of Fluids, 33(1):013601, 2021.
The credit authorship contribution corresponding to authors are as follows:
Sandeep Reddy Bukka: Writing - original draft, Methodology Development, Sup-
port on In-house code. Rachit Gupta: Writing - original draft, Investigation, Method-
ology Development, Post-processing, Data analysis. Allan Ross Magee: Supervi-
sion, Editing. Rajeev Jaiman: Conceptualization, Methodology, Analysis, Editing,
Supervision.
vii
Table of Contents
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Lay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
List of symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Deep learning for FSI . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Specific motivation . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Objectives and methodology . . . . . . . . . . . . . . . . . . . . 8
1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5 Organization of thesis . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1 Overview of high-fidelity simulations . . . . . . . . . . . . . . . 12
viii
2.2 Physics-based deep learning . . . . . . . . . . . . . . . . . . . . 13
2.3 Deep learning-based reduced order modeling . . . . . . . . . . . 16
2.4 Research gaps and novelty . . . . . . . . . . . . . . . . . . . . . 18
3 Numerical Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1 Full-order modeling for fluid-structure dynamics . . . . . . . . . 19
3.2 Reduced-order modeling . . . . . . . . . . . . . . . . . . . . . . 21
3.2.1 Projection-based reduced-order modeling . . . . . . . . . 21
3.2.2 Autoencoder-based reduced-order modeling . . . . . . . . 23
3.3 Deep learning methods . . . . . . . . . . . . . . . . . . . . . . . 24
3.3.1 Convolutional neural networks . . . . . . . . . . . . . . . 24
3.3.2 Long short-term memory networks . . . . . . . . . . . . 27
3.4 Proper orthogonal decomposition-based recurrent neural network . 29
3.4.1 Dataset preparation, training and prediction . . . . . . . . 32
3.5 Convolutional recurrent autoencoder network . . . . . . . . . . . 32
3.5.1 Dataset preparation, training and prediction . . . . . . . . 36
3.6 Snapshot-field transfer and load recovery . . . . . . . . . . . . . 37
4 Low-dimensional Analysis via Deep Learning . . . . . . . . . . . . . 44

4.1 Flow past side-by-side cylinders . . . . . . . . . . . . . . . . . . 45
4.1.1 POD-based recurrent neural networks . . . . . . . . . . . 47
4.1.2 CNN-based recurrent neural networks . . . . . . . . . . . 57
4.1.3 Discussion on predictive performance . . . . . . . . . . . 64
4.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5 Partitioned Deep Learning for Fluid-Structure Interaction . . . . . 68

5.1 Hybrid partitioned DL-ROM framework . . . . . . . . . . . . . . 69
5.2 Freely oscillating cylinder in external flow . . . . . . . . . . . . . 71
5.2.1 Limit cycle vortex-induced vibration . . . . . . . . . . . . 73
5.2.2 Non-stationary vortex-induced vibration . . . . . . . . . . 91
5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6 Three-dimensional Deep Learning for Parametric Unsteady Flows . 99

6.1 Field transfer and coarse-graining . . . . . . . . . . . . . . . . . 100
ix
6.1.1 Field transfer . . . . . . . . . . . . . . . . . . . . . . . . 100
6.1.2 Load recovery . . . . . . . . . . . . . . . . . . . . . . . 101
6.2 Deep transfer learning for DL-ROMs . . . . . . . . . . . . . . . . 101
6.2.1 Training and prediction . . . . . . . . . . . . . . . . . . . 103
6.3 Flow predictions past a sphere . . . . . . . . . . . . . . . . . . . 104
6.3.1 Unsteady flow at constant Reynolds number . . . . . . . . 105
6.3.2 Unsteady flow with variable Reynolds number . . . . . . 115
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7 Conclusions and Recommendations . . . . . . . . . . . . . . . . . . 125

7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.2 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
x
List of Tables
Table 4.1 The flow past side-by-side cylinders: Network and parameter
details for the encoder-decoder type recurrent network. . . . . 51
Table 4.2 The flow past side-by-side cylinders: Comparison of the POD-
RNN with convolutional recurrent autoencoder network (CRAN). 65
Table 5.1 An elastically-mounted circular cylinder undergoing vortex-induced

vibration: Comparison of the full-order forces and the displace-
ments in the present study and the benchmark data. CD and CL
represent the drag and lift force coefficients, respectively. Ax
and Ay represent the x and y displacements of the cylinder, re-
spectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Table 5.2 Freely vibrating circular cylinder in a uniform flow (m∗ = 10,Ur =
5, Re = 200): Network and hyperparameters details of the closed-
loop RNN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Table 5.3 Freely vibrating circular cylinder in a uniform flow (m∗ = 10,Ur =
5, Re = 200): Pixelated forces and the reconstruction accuracy
on the various snapshot DL-ROM grids with respect to the full-
order grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Table 5.4 Freely vibrating circular cylinder in a uniform flowr (m∗ = 10,Ur =
5, Re = 200): Deep CRAN layer details: Convolutional kernel
sizes, numbers and stride. Layers 6,7,8 represent the fully con-
nected feed-forward encoding, while 9,10,11 depict the similar
decoding. The low-dimensional state Ac is evolved between the
layers 8 and 9 with the LSTM-RNN. . . . . . . . . . . . . . . 86
xi
Table 5.5 Freely vibrating circular cylinder in a uniform flow: Summary
of the training and testing times of the DL-ROM components
along with force computation as compared to the full-order com-
putational time. The testing of the POD-RNN, CRAN and Al-
gorithm 3 denote the run time needed to predict p = 25 in-
stances of displacements, fields and forces, respectively. The
combined predictions denote the total run time for all these pre-
dictions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Table 6.1 The flow past a sphere: The present study’s full-order force val-
ues compared to benchmark data. CD and CL represent the
mean drag and lift force coefficients, respectively. St is the
Strouhal number. . . . . . . . . . . . . . . . . . . . . . . . . . 106
Table 6.2 Comparison of computational resources used for the 3D CRAN
and 2D CRAN training. . . . . . . . . . . . . . . . . . . . . . 110
Table 6.3 Summary of the offline and online times for 3D CRAN vs. 3D
FOM simulations. . . . . . . . . . . . . . . . . . . . . . . . . 115
Table 6.4 Summary of the offline and online times for 3D CRAN vs. 3D
FOM simulations for variable Re-based flow. . . . . . . . . . . 123
xii
List of Figures
Figure 1.1 Schematic of an autonomous digital twin and its components

[4] towards intelligent and green marine vessel (IGMV). . . . 2
Figure 1.2 High-level representation of digital twins for marine vessel.
(a) High-fidelity fluid-structure simulations of a coupled vessel
connected with riser pipeline. (b) Physics-based deep learning
for digital twin and digital marine environment. . . . . . . . . 5
Figure 1.3 Illustration of underwater radiated noise sources due to pro-
peller vibration and vessel system. (Image courtesy to UBC
Computational Multiphysics Laboratory https://cml.mech.ubc.
ca/). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Figure 1.4 Schematic of integration framework for high-fidelity simula-
tions with DL-based reduced-order modeling for data-driven
computing. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Figure 3.1 Illustration of the projection and autoencoder (AE) based reduced-
order modeling. (i) Projection-based ROM creates an orthonor-
mal basis functions (modes) for the reduced representation,
which is followed by the Galerkin projection on truncated basis
functions. (ii) Convolutional autoencoder ROM framework is
built from full-order data using nonlinear encoding and decod-
ing neural space without requiring the underlying governing
equations. Refer to the details of the variables in the main text
in section 3.2. . . . . . . . . . . . . . . . . . . . . . . . . . . 23
xiii
Figure 3.2 An abstract representation of a convolutional neural network
(CNN) architecture for reduced-order modeling. The convolu-
tion step is denoted by blue arrows. Each neural layer outputs
convolutional maps (CM) for extracting features. These maps
are further connected with fully-connected (FC) networks to
obtain the low-dimensional representation Anc from a full-order
snapshot Sn . Note that Nh << (Nx × Ny × Nz ). . . . . . . . . . 25
Figure 3.3 (a) Closed-loop and (b) encoder-decoder type recurrent neural
networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Figure 3.4 Schematic of the POD-RNN framework with a block diagram
for the iterative prediction. With one input driver yn , the pre-
dictions ŷn+1 , ŷn+2 , ..., ŷn+p are achieved autonomously. The
diagram illustrates the prediction of p number of time steps
from ŷn+1 to ŷn+p . The dashed lines imply an iterative process
of generating all the predictions between ŷn+2 and ŷn+p using
the POD-RNN method. . . . . . . . . . . . . . . . . . . . . . 30
Figure 3.5 Schematic of the CRAN framework for flow field prediction.
The encoding is achieved by reducing the input dimension from
Nx × Ny to Ac via CNNs (Conv2D) and the fully connected
networks. The decoding is achieved using the fully connected
layers and the transpose CNNs (DeConv2D). Between the en-
coding and decoding space, the LSTM-RNN evolves the low-
dimensional state Ac . . . . . . . . . . . . . . . . . . . . . . . 34
Figure 3.6 Illustration of an iterative cycle for mesh-to-mesh field transfer
and load recovery to determine the CRAN snapshot grid. See
the details of all variables in the main text. . . . . . . . . . . . 38
Figure 3.7 Identification of interface cells in the DL-ROM grid and finite
difference interpolation of field at the cell faces (green cross).
The blue nodes contain the flow field values and the red dots
are marked as zero. . . . . . . . . . . . . . . . . . . . . . . . 40
xiv
Figure 4.1 The flow past side-by-side cylinders: (a) Schematic of the prob-
lem set-up, (b) full-domain computational mesh view and (c)
close-up computational mesh view. . . . . . . . . . . . . . . 46
Figure 4.2 The flow past side-by-side cylinders: Cumulative and percent-
age of modal energies. (a)-(b) for pressure field, (c)-(d) for
x-velocity field. . . . . . . . . . . . . . . . . . . . . . . . . . 47
Figure 4.3 The flow past side-by-side cylinders: First four most energetic
time-invariant spatial POD modes along with % of total energy
for pressure field. . . . . . . . . . . . . . . . . . . . . . . . . 48
Figure 4.4 The flow past side-by-side cylinders: First four most energetic
time-invariant spatial POD modes along with % of total energy
for x-velocity field. . . . . . . . . . . . . . . . . . . . . . . . 49
Figure 4.5 The flow past side-by-side cylinders: Time histories of first
eight modal coefficients shown from 75 to 325 tU∞ /D: (a)
Pressure field and (b) x-velocity field. . . . . . . . . . . . . . 50
Figure 4.6 The flow past side-by-side cylinders: Temporal evolution (pre-
diction) of modal coefficients from 875 till 950 tU∞ /D for (a)
pressure field and (b) x-velocity field. . . . . . . . . . . . . . 52
Figure 4.7 The flow past side-by-side cylinders: RMSE for the predicted
against true modal coefficients for (a) pressure (b) x-velocity. 53
Figure 4.8 The flow past side-by-side cylinders: Comparison of predicted
and true fields (POD-RNN model) along with normalized re-
construction error En at (a) tU∞ /D = 898, (b) tU∞ /D = 923,
(c) tU∞ /D = 948 for pressure field. . . . . . . . . . . . . . . 54
and true fields (POD-RNN model) along with normalized re-
construction error En at (a) tU∞ /D = 898, (b) tU∞ /D = 923,
(c) tU∞ /D = 948 for x-velocity field. . . . . . . . . . . . . . 55
Figure 4.10 The flow past side-by-side cylinders: Predicted and actual (POD-
RNN model) pressure force coefficients. (a) Drag (Cylinder 1),
(b) lift (Cylinder 1), (c) drag (Cylinder 2), (d) lift (Cylinder 2). 56
xv
Figure 4.11 The flow past side-by-side cylinders: Mean normalized squared
error Ef for first Nt predicted time steps with respect to size of
low-dimensional state Nh of the CRAN for (a) pressure field
and (b) x-velocity field. Note that NA = Nh . . . . . . . . . . . 58
Figure 4.12 The flow past side-by-side cylinders: Comparison of the mapped
pixel force and full-order force coefficients on the training data
due to pressure for Cylinder 1 ((a)-(b)) and Cylinder 2 ((c)-(d))
using snapshot-FTLR Ψ mapping. . . . . . . . . . . . . . . 59
and true fields (CRAN model) along with normalized recon-
struction error En at (a) tU∞ /D = 898, (b) tU∞ /D = 923, (c)
tU∞ /D = 948 for pressure field. . . . . . . . . . . . . . . . . 61
and true fields (CRAN model) along with normalized recon-
struction error En at (a) tU∞ /D = 898, (b) tU∞ /D = 923, (c)
tU∞ /D = 948 for x-velocity field. . . . . . . . . . . . . . . . 62
Figure 4.15 The flow past side-by-side cylinders: Predicted and actual (CRAN
model) pressure force coefficients for all test time steps: (a)
Drag (Cylinder 1), (b) lift (Cylinder 1), (c) drag (Cylinder
2), (d) lift (Cylinder 2). Note that at a time, one input time
step (blue dot) is used to predict the next sequence of Nt = 25
steps, until a new input is fed. This helps in reducing the com-
pounding effect of the errors while still achieving predictions
in closed-loop recurrent fashion. . . . . . . . . . . . . . . . . 63
Figure 4.16 The flow past side-by-side cylinders: Comparison between the
temporal mean error Es for POD-RNN and CRAN model. (a),(b)
for pressure and (c),(d) for x-velocity. . . . . . . . . . . . . . 66
xvi
Figure 5.1 Illustration of a hybrid partitioned DL-ROM framework for
fluid-structure interaction. The field variables Ωf (t) are learned
and predicted on the uniform grid using the CRAN (blue box),
while the interface information Ωs with moving point cloud is
predicted via the POD-RNN (red box). These boxes exchange
the interface information (grey box) that couples the system
via level-set Φ and force signals F̄b . The yellow box demon-
strates the synchronized predictions. As the CRAN learns the
flow variables on the uniform grid (blue box), the level-set Φ
is utilized in calculating the pixelated force signals F̄b on the
interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Figure 5.2 Freely vibrating circular cylinder in a uniform flow: (a) Schematic
of an elastically-mounted cylinder undergoing VIV, (b) repre-
sentative full-order domain and the near cylinder unstructured
mesh. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Figure 5.3 Freely vibrating circular cylinder in a uniform flow (m∗ = 10,Ur =
5, Re = 200): (a) Cumulative and (b) percentage of the modal
energies for the ALE x-displacements. . . . . . . . . . . . . . 74
5, Re = 200): (a) Cumulative and (b) percentage of the modal
energies for the ALE y-displacements. . . . . . . . . . . . . . 75
5, Re = 200): Time history of the k = 2 modal coefficients
shown from 100 till 150 tU∞ /D for the (a) ALE x-displacements
and the (b) ALE y-displacements. . . . . . . . . . . . . . . . 76
5, Re = 200): Prediction of the modal coefficients. (a) ALE x-
displacements and (b) ALE y-displacements. . . . . . . . . . 78
xvii
5, Re = 200): Predicted and actual (a) x-position and (b) y-
position of the interface normalised by the diameter of cylinder
D. The root mean squared error between the true and predicted
is 2.17 × 10−4 and 1.08 × 10−2 for Ax /D and Ay /D, respec-
tively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5, Re = 200): Convergence of the pressure field with the num-
ber of pixels for different interpolation schemes at the time
step tU∞ /D = 125. (a)-(b) Maximum pressure values and the
respective errors, (c)-(d) minimum pressure values and the re-
spective errors. . . . . . . . . . . . . . . . . . . . . . . . . . 80
5, Re = 200): Qualitative behavior of the different interpola-
tion methods for the pressure field in the ALE reference at the
time step tU∞ /D = 125: (b) Nearest neighbor, (c) linear and
(d) piece-wise cubic type values on the snapshot 512 × 512
uniform grid with respect to (a) full-order CFD grid. . . . . . 82
5, Re = 200): Total pixelated force propagation (from 100-125
tU∞ /D) on the snapshot DL-ROM grids vs the full-order for
(a) the pressure drag coefficient CD,p , and (b) the pressure lift
coefficient CL,p on the training field. . . . . . . . . . . . . . . 83
5, Re = 200): Interface load behavior vs time (from 100-125
tU∞ /D) on the snapshot DL-ROM grids. (a)-(b) denote the
smoothen force propagation and (c)-(d) depicts recovered in-
terface force information for drag and lift coefficients. . . . . 85
xviii
5, Re = 200): Interface force prediction on the test time steps
from the predictive framework (from 225-250 tU∞ : (a) Pres-
sure drag coefficient CD,p , and (b) pressure lift coefficient CL,p .
The red lines depict the coarse-grain forces obtained from Al-
gorithm 3 and blue lines depict the corrected forces using a
denoising LSTM-RNN. . . . . . . . . . . . . . . . . . . . . 88
5, Re = 200): Comparison of the predicted and the true fields
along with the normalized reconstruction error E i at tU∞ /D =
(a) 232.5, (b) 240 (c) 249 for the pressure field. . . . . . . . . 89
5, Re = 200): Comparison of the predicted and the true fields
along with the normalized reconstruction error E i at tU∞ /D =
(a) 232.5, (b) 240 (c) 249 for the x-velocity field. . . . . . . . 90
7, Re = 200): Cumulative modal energies and the time history
of k = 2 modal coefficients shown from 100 till 150 tU∞ /D.
(a)-(b) for the ALE x-displacements, and (c)-(d) for the ALE
y-displacements. . . . . . . . . . . . . . . . . . . . . . . . . 92
7, Re = 200): Prediction of the modal coefficients and the in-
terface motion on the test data. (a) and (c) denote the predicted
and actual time coefficients of the ALE x-displacements and
ALE y-displacements, respectively. (b) and (d) denote the cor-
responding x and y position of the interface, normalised by di-
ameter of the cylinder D. With p = 25, the root mean squared
error (RMSE) between the true and predicted is 5.58 × 10−4
and 2.22 × 10−3 for Ax /D and Ay /D, respectively. With p =
200, the RMSE between the true and predicted is 9.11 × 10−4
and 4.44 × 10−3 for Ax /D and Ay /D, respectively. . . . . . . 93
xix
7, Re = 200): Recovered interface load behavior vs time shown
from 100-125 tU∞ /D using the snapshot-FTLR for (a) the drag,
and (b) the lift coefficients on the 512×512 snapshot DL-ROM
grid with respect to full-order. . . . . . . . . . . . . . . . . . 95
7, Re = 200): (a) Comparison of the predicted and the true
fields along with the normalized reconstruction error E i at tU∞ /D =
250 for the pressure field, and (b) interface drag and lift predic-
tion on the test time steps from the predictive framework with
multi-step predictive cycle p = 25. The CRAN predictions are
accurate for the first 25-30 time steps in the future, after which
the predictions can diverge. . . . . . . . . . . . . . . . . . . . 96
Figure 6.1 Schematic of a 3D mesh-to-mesh field transfer and load recov-

ery process to determine the 3D CRAN snapshot grid. Refer
to the details of all variables in section 6.1. . . . . . . . . . . 100
Figure 6.2 Illustration of the transfer learning process for training variable
Re flows. Source refers to learning and extrapolating single
Re flows in time. Target refers to learning and extrapolating
multiple Re flows in time. Note that for training steps nv < n. 102
Figure 6.3 (a) Schematic and associated boundary conditions of flow past
a sphere and deep learning domain of interest. (b) Representa-
tive CFD mesh for the entire domain sliced in Z/D = 10 plane. 106
Figure 6.4 The flow past a sphere: (a) Pressure field convergence with
number of flow voxels for interpolation scheme variants. ε(.)
is the respective relative error. (b) Descriptive behaviour of
nearest-neighbour (middle) and linear interpolation (right) tech-
niques for pressure field in the DL space 128 × 128 × 128 with
respect to CFD space (left) sliced at Z/D = 10. Plots corre-
spond to tU∞ /D = 200. . . . . . . . . . . . . . . . . . . . . . 107
xx
Figure 6.5 The flow past a sphere: Voxel interface force propagation and
load recovery effects on various snapshot 3D DL grids (shown
from 170-245 tU∞ /D). Voxel drag and lift components (Row
1). Corresponding voxel force recovery (Row 2). Red, blue
and green dashed lines represent the grid 32 × 32 × 32, 64 ×
64 × 64 and 128 × 128 × 128, respectively. The black line de-
picts the full-order force. . . . . . . . . . . . . . . . . . . . . 109
Figure 6.6 The flow past a sphere: Evolution of the loss function Eh with
training iterations for different evolver cell sizes Nh . P and
U denote the 3D CRAN trained with pressure and x-velocity
datasets, respectively. Blue and red dots depict the saved in-
stances for testing the pressure and x-velocity fields, respec-
tively. The blue cross represents the initialization of velocity
training from saved pressure parameters. . . . . . . . . . . . 111
Figure 6.7 The flow past a sphere: Predicted and true pressure field com-
parison along with normalized reconstruction error E i at tU∞ /D =
365 sliced in Z/D = (8, 10) (Row 1), Y/D = (10, 12) (Row 2),
X/D = (10, 12) (Row 3). Left, middle and right contour plots
depict the prediction, true and errors, respectively. . . . . . . . 113
Figure 6.8 The flow past a sphere: Predicted and true x-velocity field
comparison along with normalized reconstruction error E i at
tU∞ /D = 365 sliced in Z/D = (8, 9.75) (Row 1), Y/D = (10.25, 12)
(Row 2), X/D = (10.5, 12) (Row 3). Left, middle and right
contour plots depict the prediction, true and errors, respectively. 114
Figure 6.9 The flow past a sphere: Predicted and actual (3D CRAN model)
(a) drag and (b) lift force coefficients integrated from the pre-
dicted pressure field on the sphere for all 100 test time steps
with multi-step predictive sequence p = 20 and p = 100. . . . 115
Figure 6.10 The variable flow past a sphere: Recovered voxelated load
propagation (pressure drag and lift) on DL grid 64 × 64 × 64
vs full-order CFD grid for variable Re flows. Dashed and solid
lines indicate the recovered DL grid loads and full-order loads,
respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
xxi
Figure 6.11 The variable flow past a sphere: Evolution of the loss function
Eh with training iterations for different evolver cell sizes Nh . P
and U denote the 3D CRAN models trained with variable Re-
based pressure and x-velocity datasets, respectively. The blue
cross and red cross represent the initialization of pressure and
x-velocity training from optimized single Re 3D CRAN model
from Fig. 6.6. Blue and red dots depict the new saved instances
of the 3D CRAN parameters. . . . . . . . . . . . . . . . . . . 118
Figure 6.12 The variable flow past a sphere: Predicted and true pressure
field comparison with normalized reconstruction error E i at
tU∞ /D = 372.5 sliced in Z/D = 10 plane. Left, middle and
right contour plots depict the prediction, true and errors, re-
spectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Figure 6.13 The variable flow past a sphere: (a) Predicted (left) and true
(right) x-velocity field comparison. (b) Comparison of the
streamwise velocity profiles of the 3D CRAN prediction and
ground truth at three locations Y/D = 9.0, 9.5, 10 for all Reynolds
numbers. Results are plotted at test time tU∞ /D = 372.5 sliced
in Z/D = 10 plane. Circles indicate the 3D CRAN predictions,
and solid lines represent the ground truth. . . . . . . . . . . . 121
Figure 6.14 The variable flow past a sphere: 3D CRAN and FOM compari-
son of (a) mean drag and (b) mean lift variation over the sphere
for different Reynolds numbers. . . . . . . . . . . . . . . . . 122
xxii
List of symbols
ˆ
(.) All deep learning predictions
α Initial learning rate
Ax , Ay Amplitude of the streamwise and transverse vibrations
Aν , Ac POD, autoencoder time coefficients
f
b Body force on the fluid
s
b Body force on the bluff body
cs Translational damping coefficient matrix
CD , CL Drag and lift coefficients
CD,p , CL,p Pressure drag and lift coefficients
Cx , Cy , Cz Steamwise, transverse and vertical force coefficients
D Characteristic length of the bluff body
Eh Hybrid loss function
fn Translational natural frequency
fν , fc POD, autoencoder low-dimensional mapping function
Fb , FΓfs Pixelated, full-order force signals
Fs Fluid force on the bluff body
gν , gc POD, autoencoder low-dimensional evolver function
Γfs Fluid-structure interface
I Identity Tensor
xxiii
ks Translational stiffness coefficient matrix
Φ Level-set function
ms Rigid body mass
m∗ Mass ratio
µf Dynamic viscosity of the fluid
n Unit normal vector
ν POD basis modes
ntr , nts Number of training, testing time steps
ns , Ns Number of mini-batches, batches
Nh Size of the autoencoder low-dimensional state
Nt RNN evolver time steps
Ntrain Number of training iterations
Nx , Ny , Nz Number of nodes in respective Cartesian directions
Ωf Fluid domain
Ωs Solid domain
p Multi-step sequence length
pf Fluid pressure
Ψ Functional recovery mapping
Re Reynolds number
ρf Density of the fluid
S, S Field snapshot, trainable input matrix
σf Cauchy stress tensor
θevolver,ν Evolver parameters of POD-RNN
θenc,c , θevolver,c , θdec,c Encoder, evolver and decoder parameters of CRAN
uf , us Fluid velocity, bluff body velocity
U∞ Uniform inlet velocity
Ur Reduced velocity
w Mesh velocity
Y Displacement vector snapshot matrix
ζ Gaussian filter
xxiv
Glossary
3D Three-dimensional
ADAM Adaptive Moment Estimation
ALE Arbitrary Lagrangian Eulerian
CFD Computational Fluid Dynamics
CNN Convolutional Neural Network
CPU Central Processing Unit
CRAN Convolutional Recurrent Autoencoder Network
DL Deep Learning
DL-ROM Deep Learning-based Reduced-order Model
FOM Full-order Model
FSI Fluid Structure Interaction
GPU Graphics Processing Unit
HPC High Performance Computing
LSTM Long Short-term Memory
PDE Partial Differential Equation
POD Proper Orthogonal Decomposition
RMSE Root Mean Squared Error
RNN Recurrent Neural Network
ROM Reduced-order Model
Snapshot-FTLR Snapshot Field Transfer and Load Recovery
VIV Vortex Induced Vibration
xxv
Acknowledgments
I want to express my sincere and deepest gratitude to my advisor, Prof. Rajeev

Jaiman, for his unwavering support over the past two years. He has imparted a
wealth of knowledge and life lessons to me. He has given me plenty of freedom to
pursue my interests while offering the necessary guidance and inspiration when I
need it. I consider myself incredibly fortunate to have a devoted and approachable
supervisor. I cannot thank him enough for everything he has done for me.
I would also like to thank my MASc committee members, Prof. Mauricio
Ponga and Prof. Ryozo Nagamune, for participating in my research and providing
valuable inputs and constructive criticism along the way.
I am grateful to my collaborator Dr. Sandeep Reddy Bukka. Working with
him was a stimulating experience, and I would not have been in the same posi-
tion without his guidance. I would like to appreciate the support I received from
the colleagues of my research group at Computational Multiphysics Lab: Vaib-
hav, Suraj, Xiaoyu, Amir, Shayan, Indu, Wrik, Biswajeet, Jose, for the meaningful
discussions we had both academic and non-academic. Meanwhile, I am grateful
to Paul, Ruhi, Divya, Camilo, and Marcos, all of whom I met at UBC. You have
given me a lifetime of vivid memories that I will carry with me wherever I go.
I would like to acknowledge the financial support of NSERC and Mitacs that
sponsored parts of the research discussed in this thesis. I would also like to ac-
knowledge the computational resources and services provided by Advanced Re-
search Computing at UBC and Compute Canada for this research.
Above all, this thesis is dedicated to my family: my parents, grandparents, and
lovely sister. Thank you for your unconditional support, sacrifices, and love.
xxvi
Chapter 1
Introduction
Ocean shipping accounts for more than three-quarters of overall freight transport
activity and is an important enabler of international trade [5, 6]. In terms of energy
use per tonne-kilometer transported, it is the most energy-efficient mode of trans-
port [5]. However, such commercial shipping activities are known to consume
about 300 mega tonnes of fuel annually and give rise to about 3% of anthropogenic
carbon emissions [7]. In addition, anthropogenic noise arising from shipping ves-
sels is shown to adversely impact marine life in the ocean environment, particularly
marine mammals [8–11]. The main noise sources come from large commercial
vessels such as cargo ships, tankers, cruise ships and ferries as well as small size
watercrafts such as motorized boats, fishing vessels and tug boats. The noise gen-
eration from these vessels and watercrafts depends on several factors including
vessel speed, hull shape, propulsion system and transit condition [12, 13]. These
anthropogenic noise sources can influence the abilities of marine mammals in their
essential life activities (e.g., foraging, communication, echolocation, migration, re-
production) [8, 9, 14].
In response to ongoing maritime activities and sustainable future demands,
there is a growing need to develop new marine vessels and propulsion systems 1
2. The vessels are targeted to protect the environment by improving efficiency and
1 https://green-marine.org/wp-content/uploads/2020/09/1-Abigail Fyfe.pdf
2 https://www.portvancouver.com/wp-content/uploads/2021/04/2021-04-05-ECHO-2020-Annual-report
Final-1.pdf
1
System
Control Sensors
Digital
Twin
Prediction Data
DL-ROM
Figure 1.1: Schematic of an autonomous digital twin and its components [4]
towards intelligent and green marine vessel (IGMV).
4
reducing carbon/noise pollution caused by shipping in open waters. Advances in

multiphysics predictions and data science, efficient structural design and optimiza-
tion, innovative sensors and control techniques can play vital roles in reducing car-
bon/noise pollution by developing next-generation of vessel design and operation
[15–17]. Along with simplified theory/semi-empirical methods, model testing and
sea trials, advanced simulation tools based on computational fluid dynamics (CFD)
and finite element analysis (FEA) are routinely employed by the marine industry
for the development of quieter and efficient technologies [18]. Towards cleaner
and quieter transportation, the marine and offshore industries are undergoing the
Industrial 4.0 digitalization for automation and autonomy. While automation refers
to activities performed without human intervention, autonomy refers to a system’s
ability to achieve objectives while operating independently of external control.
Digital twin technology [4, 19] has emerged as a revolutionary concept for next
generation transport vehicles and power/energy generation systems. A digital twin
is a virtual representation that serves as the real-time digital counterpart of a physi-
2
cal object or process. While digital twin technologies are rapidly evolving, the ma-
rine and offshore community must combine them with physics-based deep learning
technologies [16, 20] to develop intelligent and green marine vessels (IGMVs). In
addition, sensor technology and structural health monitoring have seen remarkable
growth over the last decade. Information gathered through the application of mul-
tiple sensors onboard a ship can be used to increase energy efficiency and safety
by controlling noise and pollution. Figure 1.1 depicts a typical infrastructure of
a marine digital twin as well as its major components. Such autonomous digital
twin systems require the integration of high-dimensional data with reduced-order
models (ROMs) and deep learning (DL). This unique integration paves way for
the design of next-generation of IGMVs, offering crucial real-life data to decrease
uncertainties and improve efficiency.
Completely aligned with the above requirements and targets, this dissertation
aims to provide tools and techniques for constructing a digital twin for marine ves-
sel operation. For the development of a digital twin which can evolve nearly in
real-time and emulate its physical system counterpart, one of the main difficul-
ties concerns the handling of nonlinear multiphysics and fluid-structure interaction
(FSI) effects with efficient and reliable predictions. Of particular interest is to de-
velop a new simulation framework integrating high-fidelity physics-based simula-
tions with deep learning-based reduced-order models (DL-ROMs) for data-driven
predictions. These advanced tools and techniques are precisely aligned with the
Industry 4.0 trends of automation and autonomy while fostering the development
of IGMVs.
1.1 Deep learning for FSI

Marine and offshore engineering structures such as vessels, propellers, oil and gas
platforms are designed to operate in fluid environments. Additionally, the inter-
action of fluid flow with the structures results in a coupled physical phenomenon
of fluid-structure interaction (FSI). FSI plays a vital role in many multiphysics
problems, and marine/offshore applications are no exception. Some canonical ex-
amples of FSI include flow past oscillating hydrofoil and ocean currents on subsea
pipelines, risers, and moorings [21]. The physics of FSI is based on the formula-
3
tion of appropriate partial differential equations (PDEs) that describe underlying
dynamics. The equations are a set of nonlinear PDEs that produce a variety of
dynamics, including resonance, non-stationary oscillation, and symmetry breaking
flows. Closed-form analytical solutions of such nonlinear PDEs are practically im-
possible to acquire for typical engineering systems with complicated geometries.
As a result, we frequently use high-performance computing to examine such sys-
tems. Accurate solutions have been possible by solving millions of variables using
state-of-the-art numerical techniques such as the finite element method. In the
recent decade, hardware and computational power have increased tremendously,
leading to the creation of vast volumes of data. PDEs, on the other hand, have
various challenges, including high-dimensionality and nonlinearity, which limit
real-time predictions and control efforts. With the recent advancements in DL and
artificial intelligence (AI), learning and making inference from data have taken the
spotlight.
DL [22, 23] is a subset of machine learning that refers to the use of multilayered
neural networks to learn input-output datasets and make predictions by learning a
set of training data. By learning the underlying data-driven input-output model via
deep neural networks, the goal is to make predictions for unseen input data. Using
deep neural networks along with a collection of algorithms, one can find useful fea-
tures or embedding functions from the input-output relation in datasets. Deep neu-
ral networks have the ability to automatically extract functional relations from data
with hierarchical importance. On the other hand, however, the biggest shortcoming
of DL in physical simulations is obvious: by the very design, DL algorithms are
black-box models. They provide immense expressive power but at the cost of no
physical interpretability. This severely limits the application of a purely DL-based
data-driven model to extrapolate and generalize to multiphysics problems such as
in marine and offshore applications. According to the free lunch theorem [24], it is
difficult to find a single algorithm that is best suited for learning all possible data
sets or scenarios without any prior domain knowledge or information.
It is therefore of paramount importance to leverage any prior knowledge about
the data or the context of the problem when choosing a machine learning algorithm.
In traditional machine learning, this has been achieved by adjusting the neural ar-
chitecture designs. For example, the convolutional neural networks (CNNs) pre-
4
(a)
Train
Predict
(b)
Figure 1.2: High-level representation of digital twins for marine vessel. (a)
High-fidelity fluid-structure simulations of a coupled vessel connected
with riser pipeline. (b) Physics-based deep learning for digital twin and
digital marine environment. 5
serve translational invariance of an image while recurrent neural networks (RNNs)

retain sequential information of language data. On a similar note, for any dynam-
ical system described by PDEs, the underlying physical laws constitute some set
of constraints that should be applied to the DL model, for example, reduced-order
modeling. Enforcing these constraints will improve the learning of the DL-based
data-driven model in terms of accuracy, convergence, interpretability and general-
ization to unseen scenarios. There can be numerous ways to enforce such physical
constraints in DL and is a problem-dependent non-trivial task. Such modification
of DL with physical constraints is termed as physics-based deep learning and is
described in detail in Chapter 2. In this dissertation, we constrain DL with reduced-
order modeling for efficient predictions of unsteady flows and FSI. This paves way
for the development of digital twins for marine and offshore applications. Fig.
1.2(a) depicts such an illustration of full-order FSI simulation around marine ves-
5
sel while physics-based deep learning digital twin is shown in Fig. 1.2(b).
1.2 Specific motivation

The majority of marine and offshore structures are represented as bluff body ge-
ometries. Moreover, rotating propeller blades at a high angle of attack can behave
like bluff bodies with massive separation and wake flow. At high angle of attacks,
a relatively long aspect ratio of blades can be represented as cylinders (2D ge-
ometry) while the wake dynamics of short aspect ratio blades can be modeled as
spheres (3D geometry). Multi-column prismatic and cylindrical systems are also
quite common in the offshore industry. Freestream flow past such structures results
in a generation of a wide range of vortex and wake patterns. These unsteady flow
patterns can excite the immersed structure and can have sustained vibrations via
the lock-in phenomenon. During lock-in, the frequencies of the wake/vortex sys-
tem come close to the natural frequencies of the underwater structure which give
rise to vortex-induced vibration (VIV). For instance, propeller blades are rotating
wing surfaces that produce thrust to propel marine vessels. Depending on operating
conditions, they experience VIV due to unsteady vortex shedding and cavitation in
several different locations e.g., tip vortex and hub vortex. This can lead to flow-
generated noise. In nearly all problems of flow-generated noise, the energy source
for sound production is some form of flow unsteadiness or flow-induced vibration
[25, 26].
When the natural frequency of propeller blades coincides with the vortex shed-
ding frequency, it can give rise to the propeller singing phenomenon [27, 28]. The
propeller singing phenomenon is defined as the resonance between the local nat-
ural frequency of the propeller blade tip and the vortex shedding frequency at the
trailing edge of the blade. Propeller singing makes very intensive levels of noise
and is amongst the major sources of underwater radiated noise (URN). Hull vibra-
tion and onboard machinery are among other sources of URN. Fig. 1.3 highlights
the URN sources and propeller singing for the marine vessel and unsteady vortex
system. One of the crucial aspects of structural design and operation is identi-
fying the flow/structural parameters that lead to relatively large structural motion
and propeller singing due to VIV. Cavitation [29] can further increase noise and
6
Vibrating
Element
Vibration
transmitted URN due to vortex created by
to structure
hull forms and appendages
URN due to structure vibration

creating pressure waves
Marine life and environment
URN peaks
URN due to propeller
due to gear mesh
crating pressure waves
5
Figure 1.3: Illustration of underwater radiated noise sources due to propeller

vibration and vessel system. (Image courtesy to UBC Computational
Multiphysics Laboratory https://cml.mech.ubc.ca/).
reduce propulsion efficiency, as well as introduce the unwelcome risks of longer-

term propeller damage. With noise levels from large vessels in the frequency range
of 5–1,000 Hz predominating, large ocean areas are affected and these frequencies
overlap the audible ranges used by many marine mammals. As the propeller is
one of the main sources of underwater noise generated by ships, it is important to
predict and control its underwater noise characteristics. In this respect, the fluid-
structure interaction of a rotating propeller blade [30] is fundamental to understand
the frequency lock-in and propeller singing. While advanced CFD-based models
provide reliable solutions and physical insight for propeller singing and VIV [31],
these 3D models are very time consuming to create and analyze and need a highly
evolved design. As a result, the development of a surrogate physics-based deep
learning toolbox for VIV and unsteady flows becomes an integral part of research
for fast and safe operations of marine/offshore systems.
As the primary motivation of this study, we consider the nonlinear dynamical
system of complex 3D unsteady wake flow and FSI of the bluff body for data-
7
driven prediction. While physics-based techniques have revolutionized the perfor-
mance of engineering systems, they pose difficulties for efficient optimization and
real-time predictions required by the digital twin development. As ground truth
data, high-fidelity simulations provide useful information. However, the vast size
of databases on the other hand creates significant analysis challenges. Further-
more, much of the data generated by full-order physics-based tools goes unused;
for example, data from one simulation is not used to reduce the processing cost of
subsequent simulations. These drawbacks of physics-based simulations have in-
creased the demand for innovative data-driven techniques alongside DL, which are
primarily motivated by the need for fast and accurate digital twins.
This dissertation addresses the key elements of reduced-order modeling via DL
by utilizing high-fidelity data leading to a series of deep learning-based reduced-
order models (DL-ROMs). The DL-ROM techniques for FSI and 3D unsteady
problems are described and applied for data-driven predictions.
1.3 Objectives and methodology

The primary objective of this study is to investigate the data-driven analysis and
prediction of VIV and 3D unsteady flows via DL-ROMs. The current work is di-
vided into two parts to achieve the aforementioned objectives. The first part deals
with an overall framework for constructing DL-ROMs to solve the nonlinear dy-
namical prediction of unsteady flow. The focus is to investigate the predictive per-
formance of hybrid DL-ROMs, which vary in obtaining the low-dimensional fea-
tures, i.e., proper orthogonal decomposition (linear approach) and convolutional
autoencoders (nonlinear approach). The second part of this study includes the
model reduction strategies and prediction of VIV and 3D unsteady flows. The
application of the knowledge gained in the previous parts is utilized for the devel-
opment of partitioned, coupled, scalable and adaptable DL-ROM methodologies
for unsteady flows with moving interfaces and 3D parametric effects.
The methodology using numerical discretizations of PDEs for high-fidelity
simulations can be termed as forward problem. On the other hand, data-driven
techniques using reduced-order modeling and DL can be collectively termed as
inverse problem. Inverse problems are based on input-output data and can deal
8
HPC-CFD Full-order data Data Science
3D FSI
Domain NVIDIA Tesla
V100 GPU
Field Data Low-order projection
Deep Neural Database ROM approximation

Network
DL-based
ROM
Features ROM modeling
Re-Train DNN
Prediction Add DNN to Database

Yes
Flow predictions Parametric forces

Data-driven
Converged?
y computing
z x
No
Figure 1.4: Schematic of integration framework for high-fidelity simulations 5
with DL-based reduced-order modeling for data-driven computing.
with the computational cost and complexities of solving forward problems. Fig.
1.4 shows a high-level overview of the integrated procedure to combine high-
fidelity simulations with reduced-order modeling and DL. The framework begins
with solving the forward problem on a high-performance computer and collecting
the full-order data. The next step involves data-driven modeling. For general data-
driven modeling, two advanced DL-ROM frameworks are utilized that are based
upon projection-techniques and autoencoder to extract low-dimensional features
from full-order snapshots. The low-dimensional features are evolved over time us-
ing RNNs. The projection-based DL-ROM provides a linear ROM using the proper
orthogonal decomposition (POD) that gives a low-order approximation of the un-
steady flow dynamics. The obtained low-dimensional states are advanced in time
using RNNs. The hybrid model is termed as the POD-RNN. However, for complex
unsteady flows with moving interfaces and 3D geometries, the only way to reach
this aim is with complete nonlinear model reduction techniques. To demonstrate
9
this idea, we employ the convolutional recurrent autoencoder network (CRAN).
This represents a DL-ROM method with the structure of a nonlinear state-space
format to build a data-driven model for forecasting VIV in lock-in and 3D paramet-
ric flow predictions. Mathematical details of the integrated procedure are further
discussed in Chapter 3. Here are some of the key questions and goals that belong
to this part of the study:
• Can we construct an efficient DL-ROM for data-driven time series predic-

tions of general nonlinear dynamical systems?
• How can we exploit and combine various DL-ROMs for constructing physics-
based digital twins involving complex FSI and 3D unsteady flow effects?
• Can we generalize the effects of parameters (e.g., Reynolds number) for

data-driven predictions via hybrid DL-ROMs?
1.4 Contributions
The contributions in this dissertation help to advance the field of computational
fluid dynamics by using data-driven and machine learning approaches. This re-
search spans a wide range of topics, including fluid mechanics, computational
physics, data science and deep learning. Some of the key contributions from this
study can be summarized as:
• Data-driven study on unsteady flow predictions for complex wake interfer-

ence in the multi-body system via detailed low-dimensional analysis and DL.
• Generalized partitioned and modular DL-ROM approach to couple the fluid

and solid subdomains via deep neural networks.
• Efficient time series predictions of FSI for a freely vibrating structure in

external flow.
• Iterative and optimal low-resolution structure grid search algorithm for CNNs
based on unstructured fluid grid.
• Effective learning and inference capabilities of DL-ROM for unsteady flow

past a 3D sphere geometry.
10
• Analysis of deep transfer learning to reduce training time and hyperparame-
ter search for DL-ROMs.
• Generalized 3D data-driven DL-ROM for parametric flow and force predic-

tions with variable Reynolds number.
1.5 Organization of thesis

The thesis is organized as per the contributions listed in the previous section. In
Chapter 2, a detailed literature review on high-fidelity modeling, physics-based
deep learning, and formulation of DL-ROMs are presented. Chapter 3 presents the
numerical methodology used throughout the dissertation. It starts with the full-
order and reduced-order modeling of FSI, then moves to DL techniques including
convolutional and recurrent neural networks. This chapter also presents the DL-
based ROM frameworks and data processing utilities in detail.
Chapter 4 focuses on the data-driven analysis of two-hybrid DL-ROMs, the
so-called POD-RNN and CRAN, for flow past static bluff bodies with emphasis
on multi-body systems. A detailed data-driven assessment is provided in terms of
computational efficiency and accuracy. In Chapter 5, a partitioned DL-ROM ap-
proach is developed for the first time to couple the fluid and solid subdomains via
deep neural networks. In particular, the attributes of the POD-RNN and the CRAN
frameworks are unified for predicting the coupled FSI dynamics. The data-driven
analysis of the VIV motion of a freely oscillating cylinder is thoroughly studied.
Chapter 6 presents a 3D DL-ROM for the time series prediction of unsteady flow
past a sphere. The primary goal is to study parametric flow and force predictions
with variable Reynolds numbers. For this purpose, a 3D convolutional recurrent au-
toencoder process with transfer learning is used for learning complex 3D unsteady
wake past a sphere. Finally, some concluding remarks and recommendations for
future research are presented in Chapter 7.
11
Chapter 2
Literature Review
In this chapter, the first section establishes the importance of high-fidelity fluid-
structure interaction techniques and their problems via a substantial amount of lit-
erature. Following that, we detail the applicability of deep learning in physical
simulations for data-driven modeling via a recent literature survey. We bring a
broad classification in the growing research area of bringing inherent inductive or
physical bias in deep learning. Next, we focus on the deep learning-based reduced-
order models (DL-ROMs) as physics-based deep learning technique and their ad-
vantages through literature review. In the last section, we have stated clear gaps
in the literature on developing DL-ROMs for moving interfaces and 3D unsteady
flows. In this aspect, the novelty of the dissertation is highlighted.
2.1 Overview of high-fidelity simulations

Fluid-structure interaction is a coupled physical phenomenon that involves a mu-
tual interplay and bidirectional interaction of fluid flow with structural dynamics.
This phenomenon is ubiquitous in nature and engineering systems such as flutter-
ing flags [32], flying bats [33], offshore platforms and pipelines [34, 35], oscil-
lating hydrofoils with cavitation [29], two-phase flow in a flexible pipeline [18]
and among others. For example, the two-way coupling between the fluid and
solid exhibits rich flow dynamics such as the wake-body interaction and vortex-
induced vibrations [36], which are important to understand from an engineering
12
design or a decision-making standpoint. Due to the complex characteristics of the
fluid-structure coupling, frequent but reliable practice is to model these complex
interplay and underlying dynamics by solving numerically a coupled set of partial
differential equations (PDEs) describing the physical laws. Of particular interest in
these numerical techniques for unsteady fluid-structure interaction is to accurately
simulate the wake-body interaction in terms of vortex-induced loads and structural
displacements, which represent the prominent coupled dynamical effects.
Fluid-structure systems can undergo highly nonlinear interactions involving
complicated interface dynamics and a wide range of spatial and temporal scales.
Moreover, these complex spatial-temporal characteristics are very sensitive to phys-
ical parameters and geometric variations. Numerical techniques such as the arbi-
trary Lagrangian-Eulerian [34, 37], level-set [38], immersed boundary [39], ficti-
tious domain [40] and phase-field modeling [18, 41] can provide high-fidelity PDE-
based numerical solutions and physical insights of the underlying FSI phenomena.
Using the state-of-the-art discretization techniques such the finite element method,
accurate solutions have been possible by solving millions of fluid and structural
variables using the full-order models and incorporating proper treatment of the
fluid-structure interface [33]. These techniques primarily involve solving the un-
steady Navier-Stokes equations in a 3D computational domain for various geome-
tries and boundary conditions. As the complexity of an FSI system elevates, the
accurate treatment of the interface is directly linked with increasing fidelity near the
fluid-structure interface, which implies solving more and more unknown variables
numerically [18]. Furthermore, the analysis of flow involving three-dimensional
geometries involves complex and high-dimensional mesh generation. At higher
resolutions of space and time, the equation-based forward simulations can come at
the expense of prohibitively large computing time and high-dimensionality, render-
ing them almost ineffective in real-time predictions, control or multi-query analysis
required by digital twin development.
2.2 Physics-based deep learning

The applicability of deep learning has emerged as a promising alternative for con-
structing data-driven prediction models for fluid flow [17, 42] and nonlinear dy-
13
namical systems e.g., fluid-structure interaction [43]. Deep learning is a sub-
set of machine learning that refers to the use of multilayered neural networks to
classify spatial-temporal datasets and make inference from a set of training data
[22, 23, 44]. By learning the underlying data-driven input-output model via deep
neural networks, the goal is to make predictions to unseen input data. The con-
struction of heavily over-parametrized functions by deep neural networks rely on
the foundations of the Kolmogorov–Arnold representation theorem [45] and the
universal approximation of functions via neural networks [46–48]. Using deep
neural networks along with a collection of algorithms, one can find useful features
or embedding functions in a low-dimensional space from the input-output rela-
tion in datasets. Deep neural networks have the ability to separate spatial-temporal
scales and automatically extract functional relations from high-dimensional data
with hierarchical importance. Bestowed by the state-of-the-art back-propagation
and stochastic gradient descent techniques for estimating weights and biases adap-
tively, deep neural networks can provide efficient low-dimensional representation
in a flexible way while learning multiple levels of hierarchy in data [22]. While
deep neural networks are heavily overparametrized, they have an inherent bias to
induce and make inferences to unseen data which is termed as inductive bias [49].
For example, convolutional neural networks (CNNs) possess an implicit inductive
bias via convolutional filters with shared weights (i.e., translational symmetry) and
pooling to exploit scale separation [49]. Nevertheless, these black-box deep learn-
ing techniques do not account for prior domain knowledge that can be important
for interpretability, data efficiency and generalization.
In the last few years, there is a growing interest to exploit the inherent in-
ductive bias in deep learning and to infuse explicit bias or domain knowledge in
the network architectures for efficient predictions and interpretability [50]. In that
direction, there have been many promising approaches established in the research
community for a synergistic coupling of the deep learning and physical-based mod-
els [51]. These models are often trained to represent a full or partial parametriza-
tion of a forward physical process while emulating the governing equations. These
coarse-grained inference models and surrogate representations aim to reduce the
high computational costs in forecasting dynamics and to model the quantities of
interest. For example, one can use the trained parameters to carry out the state-
14
to-state time advancements [52–54] and inverse modeling [55–57]. Here, we refer
to the state-to-state time advancements as inferring dependent physical variables
from the previous states. At the same time, inverse modeling identifies the phys-
ical system parameters from the output state information. One can also exploit
the approximation properties and inductive bias of neural calculus to solve the
underlying differential equations within a deep learning framework [58–61]. With-
out appropriate modifications of the state-of-the-art deep learning techniques, the
approximate solution of the PDEs via black-box deep learning approaches may
lead to slow training, reduced accuracy, and a lack of generality to multiphysics
and multiscale systems involving complex geometries and boundary conditions
[42, 62–65]. For increasing generality, we can broadly classify the physics-based
machine learning into three categories: (a) the modification of the objective or loss
function by adding a regularizer, (b) the adjustment of neural architecture designs,
and (c) the hybridization of deep learning with projection-based model reduction.
In the first category, one can infuse the physical information by applying a
regularizer to the standard loss function. Such regularizers are generally based
on physical conservation principles or the governing equations of the underlying
problem. Many researchers have explicitly enforced such regularizers to boost
generalizability; for instance Karpatne et al. [66], Raissi et al. [67], Zhu et al.
[68], Erichson et al. [69], Geneva et al. [70], Mallik et al. [71], and among oth-
ers. Wang et. al [72] also employed physics-informed machine learning to model
the reconstruction of inconsistencies in the Reynolds stresses. For physically con-
strained neural architectures, as the second category, Daw et al. [73] and Chang
et al. [74] modified the recurrent neural network (RNN) cell structure by interme-
diate constraints while Muralidhar et al. [75] and Ruthotto and Haber [76] took a
similar route for the CNNs. Recently, Li et al. [77] developed the Fourier neural
operators by applying a Fourier transform and linear weight operator on the input
before the activation function in a neural network. CNNs have been utilized for the
prediction of steady laminar flows [78] and the bulk quantities of interest [17] for
bluff bodies. Similarly, Lee et. al [79] employed CNNs and generative adversarial
networks with and without loss function modification to predict recursive unsteady
flow. For the prediction of unsteady flow over a cylinder and airfoil, Han et al. [80]
constructed a convolutional long-short term memory network. To estimate the drag
15
force, Ogoke et al. [81] developed a graph-based convolutional network by depict-
ing the flow field around an airfoil using unstructured grids. Likewise, Snachez et
al. [82] and Pfaff et al. [64] utilized graph neural networks to represent the state
of a physical system as nodes in a graph and compute the dynamics by learning
message-passing signals. The third category involves the development of the hy-
brid DL-ROMs that take into account the spatial and temporal domain knowledge
in neural networks. The spatial knowledge is incorporated by effectively build-
ing the low-dimensional states using the projection-based techniques which inherit
physical interpretation as discussed in Miyanawala & Jaiman [16]. The temporal
knowledge comes from evolving these low-dimensional states in time. We refer
to such DL-ROM frameworks as physics-based because they incorporate physical
interpretability via proper orthogonal decomposition (POD) and its variants. These
DL-ROMs can operate in synchronization with the full-order models to boost pre-
dictive abilities.
2.3 Deep learning-based reduced order modeling

For addressing issues of high-dimensionality, reduced-order models are constructed
for low-dimensional analysis and have been widely investigated to identify domi-
nant flow patterns and make dynamical predictions. One of the principal tools is
to project a high-dimensional dataset onto an optimal low-dimensional subspace
either linearly or nonlinearly to reduce spatial dimension and extract flow features.
These low-dimensional analyses can provide essential flow dynamics for opera-
tional decision and efficiency improvement. Various projection techniques such
as POD [83], dynamic mode decomposition [84], balanced-POD [85] and Koop-
man operators [86] have been extensively studied for the dimensionality reduc-
tion, control and mode decomposition of field dataset into relevant features. The
mode decomposition can be considered as a mathematically optimal linear repre-
sentation of the flow field and can provide interpretable analysis on flow features.
For instance, Miyanawala and Jaiman [87] showcased that for flows involving low
Reynolds number, the POD modes represent one of the large-scale flow features
such as vortex shedding, shear layer or near-wake bubble.
However, projection-based reduced-order models can pose difficulty in the
16
dimensionality reduction for complex flow patterns and hyperbolic PDEs as the
number of required modes increases significantly. Instead, neural network-based
autoencoders [88, 89] are explored as an alternative for nonlinear approximation
because of their ability to automatically encode flow datasets and address some of
the limitations of linear projection techniques. Using encoder-decoder networks
and activation functions, autoencoders allow to learn nonlinear relations between
the input and the output dataset. In contrast to the projection-based reduced-order
models, autoencoders provide larger compression and a greater flexibility for the
dimensionality reduction of the data arising from the Navier-Stokes equations. Au-
toencoders have been employed in a variety of fields such as object detection [90],
sensor data analysis [91] and biometric recognition [92] due to their ease of imple-
mentation and low computational cost. For autoencoder and its variants, one can
refer to a review work of Dong et. al [93].
To achieve data-driven prediction of dynamical problems using projecting meth-
ods, many researchers combine reduced-order model spaces with deep learning
to enhance predictive abilities which can be termed hybrid DL-ROMs. Such hy-
brid architectures consider spatial-temporal domain knowledge and achieve data-
driven time series predictions. Recently proposed POD-based DL-ROMs are the
POD-CNN by Miyanawala and Jaiman [16], the POD-RNN by Bukka et al. [15],
the POD-enhanced autoencoders by Fresca and Manzoni [94]. Notably, for the
first time, Miyanawala & Jaiman [16] proposed a hybrid DL-ROM technique for
fluid-structure interaction to combine the POD and CNN to take the advantage
of the optimal low-dimensional representation given by POD and the multiscale
feature extraction by the CNN. Besides the prediction of unsteady flow and wake-
body interaction, such hybrid DL-ROM models have also been explored for other
problems including turbulent flow control application [95], bifurcating flow phe-
nomenon [96], combustion [97] and parametric steady-state solutions of the PDEs
[98, 99]. These works reported substantial speed-ups using the projection-based
deep learning models during the online predictions compared with their full-order
counterpart.
17
2.4 Research gaps and novelty
Using physics-based deep learning, the above works report substantial speed-ups
during the online predictions compared with their full-order counterpart. How-
ever, most of these works are centered around two-dimensional geometries with-
out fluid-structure interaction [78, 100–103]. Hence, there is a need to develop
a systematic and realistic data-driven framework to predict nonlinear dynamical
systems required by digital twin development.
In this dissertation, we develop deep learning-based reduced-order models for
(a) complex unsteady flow predictions and fluid-structure interaction, (b) paramet-
ric flow and force predictions with variable Reynolds number, and (c) data-driven
computational speed-ups for both offline training and online prediction of com-
plex 2D and 3D wake flow. All the developed techniques and novel algorithms
aim to address the curse of spatial dimensionality while achieving data-driven pre-
dictions. Although there are a few studies that use CNNs and autoencoder for
parametric dependent flow problems [79, 94], there is no work that attempts to
develop DL-ROM methodologies for the flow past static and moving geometries
in a way that can provide an effective mean to couple with real-time flow field
snapshots [104, 105]. The proposed DL-ROM frameworks presented in this thesis
are modular, adaptable, and entirely data-driven; they align with the digital twin
development for unsteady FSI and 3D flow effects.
18
Chapter 3
Numerical Methodology
In this chapter, we start by describing the full-order equations of a coupled fluid-

structure interaction which is followed by a description of reduced-order modeling
using projection-based ROMs and convolutional autoencoders. We then present
the mathematical formulation of the convolutional and long short-term memory
networks. Following these formulations, we detail two hybrid deep learning-based
reduced-order methodologies called the POD-RNN and the CRAN for time se-
ries prediction. The last part of this chapter includes a novel strategy termed the
snapshot-field transfer and load recovery (snapshot-FTLR) to select the structured
grid for the convolutional neural networks.
3.1 Full-order modeling for fluid-structure dynamics

This section starts by describing the full-order equations of a coupled fluid-structure
interaction. A coupled fluid-solid system in an arbitrary Lagrangian-Eulerian (ALE)
reference frame is modeled by the isothermal fluid flow interacting with a rigid
body structure:
∂ uf
ρf + ρ f (uf − w) · ∇uf = ∇ · σ f + bf on Ωf (t), (3.1)
∂t
∂ ρf
+ ∇ · (ρ f uf ) = 0 on Ωf (t), (3.2)
∂t
∂ us
ms + cs us + ks (ϕ s (z0 ,t) − z0 ) = Fs + bs on Ωs , (3.3)
∂t
19
where superscripts f and s denotes the fluid and structural variables, respectively. In
the fluid domain Ωf , uf and w represent the fluid and mesh velocities, respectively
and bf is the body force, and σ f is the Cauchy stress tensor for a Newtonian fluid
written as σ f = −pf I + µ f ∇uf + (∇uf )T . Here, pf is the pressure in the fluid, I

is the identity tensor and µ f is the fluid viscosity. Any arbitrary submerged rigid
body Ωs experiences transient vortex-induced loads and, as a result, may undergo
large structural motion if mounted elastically. The rigid body motion along the
two Cartesian axes is modeled by Eq. (3.3), where ms , cs and ks denote the mass,
damping and stiffness matrices, respectively. us represents the rigid body motion at
time t with Fs and bs as the fluid traction and body forces acting on it, respectively.
Here, ϕ s denotes the position vector that transforms the initial position z0 of the
rigid body to time t. This coupled system must satisfy the no-slip and traction
continuity conditions at the fluid-body interface Γfs (t) as follows:
uf (t) = us (t) , (3.4)

Z
σ f · ndΓ + Fs = 0, (3.5)
Γfs (t)
where n and dΓ denote the outward normal and the differential surface area of the
fluid-solid interface, respectively. While Eq. (3.4) enforces the velocity continuity
on the moving interface, the balance of the net force exerted by the fluid on the rigid
body is given by Eq. (3.5). Due to the body-conforming Eulerian-Lagrangian treat-
ment, the interface conditions provide accurate modeling of the boundary layer and
the vorticity generation over a moving body. The coupled differential equations in
Eqs. (3.1)-(3.3) are numerically solved using the Petrov–Galerkin finite element
and the semi-discrete time stepping [34]. The weak form of the incompressible
Navier-Stokes equations is solved in space using equal-order isoparametric finite
elements for the fluid velocity and pressure. We employ the nonlinear partitioned
staggered procedure for the stable and robust coupling of the fluid-structure inter-
action [34]. This completes the description of the coupled full-order FSI system.
Similarly, for unsteady flows with no FSI, the numerical discretization is achieved
using the finite element discretization of Eqs. (3.1)-(3.2) with no mesh velocity.
In relation to the model-order reduction, the coupled equations (Eqs. (3.1)-
20
(3.3)) can be written in an abstract state-space form as
dz
= F(z), (3.6)
dt
where z ∈ RM is the state vector for a coupled FSI domain with a total of M vari-
ables in the system. For a fluid-body system, the state vector involves the fluid
velocity and the pressure as z = {uf , pf } and the structural velocity includes the
three translational degrees-of-freedom. Note that the pressure p f can be written as
some function of density ρ f via the state law of a fluid flow. The right hand side
term F represents a dynamic model and can be associated with a vector-valued dif-
ferential operator describing the spatially discretized PDEs in Eqs. (3.1)-(3.3). The
temporal term dz/dt is the system dynamics which determines the instantaneous
physics of a fluid-structure system and is useful for creating a low-order represen-
tation. The resultant spatial-temporal dynamics of the fluid-structure interaction
are driven by the inputs such as the modeling parameters and boundary conditions.
Next, we review a data-driven reduced-order modeling based on traditional projec-
tion and convolutional autoencoder techniques.
3.2 Reduced-order modeling

From the perspective of a data-driven approach, the idea is to build the differen-
tial operator F by projecting onto a set of low-dimensional trial subspace. In that
sense, one can decompose F(z) to encapsulate the constant term C, a linear Bz and
nonlinear F′ (z) dynamical components as
F(z) = C + Bz + F′ (z). (3.7)
3.2.1 Projection-based reduced-order modeling

Using the Galerkin-based ROMs, we represent the state vector z ∈ RM via a sub-
space spanned by the column vectors of the low-dimensional modes V ∈ RM×K ,
with K << M. The matrix V is referred to as the reduced basis spanning the sub-
space onto which the dynamics is projected. This allows to approximate the state
21
vector z as V z̃ with z̃ ∈ RK and thereby reducing the system dynamics as
d z̃
= V T C + V T BV z̃ + V T F′ (V z̃). (3.8)
dt
Here, the reduced-order space is constructed by defining a choice of modes V ∈

RM×K using the snapshot matrix Z = {z1 z2 . . . zS } ∈ RM×S , with M number
of variables and S denotes the number of snapshots. The columns of the matrix
V = {V 1 V 2 . . . V K } form an orthonormal basis of Z̃ = {z̃1 z̃2 . . . z̃S } ∈ RK×S ,
a K dimensional subspace of RM . The linear expansion of the state vector snap-
shot matrix Z = V Z̃ allows the reduction to take place through special choices of
subspace Z̃ for any dataset. For example, the well known POD derives this sub-
space to be such that the manifold variance is preserved as much as possible when
projected to Z̃, given a fixed dimension constraint K. Using the singular value
decomposition (SVD) decomposition, the snapshot matrix can be expressed as
Z = V ΣW T = ∑KJ=1 σJ vJ wTJ , where vJ are the POD modes of the snapshot matrix
Z and Σ ∈ RK×K is a diagonal matrix with diagonal entries σ1 ≥ σ2 ≥ . . . ≥ σK ≥ 0.
The total energy contained in each POD mode vJ is given by σJ2 . V and W are
the matrices with orthonormal columns that represent the eigenvectors of ZZT and
ZT Z, respectively.
It is well known that the above POD-Galerkin process is very effective for
dimensionality reduction of the linear term while the nonlinear term cannot be
reconstructed properly for the unsteady wake flow dynamics. Therefore, the lin-
ear POD method may result in a similar order of computational expense to the
full-order simulation. For a low-order representation of such nonlinear terms, hy-
perreduction techniques such as the discrete empirical interpolation method [106]
and energy-conserving sampling and weighting [107] method can provide an ad-
ditional level of approximation for the dimensionality reduction. These hyperre-
duction strategies can reduce the required number of modes, hence decreasing the
computational cost while capturing the nonlinear regions properly [87]. All these
techniques primarily aim to resolve the nonlinear term, i.e, V T F′ (V z̃), where z̃ is
the reduced-order state of full-order state z. We note that these decomposition tech-
niques are dependent on the choice of dataset, parameters and the initial condition
of a nonlinear system. For instance, it has been shown that for the flow problems
22
Projection and autoencoder-based ROM
Snapshots POD modes
Truncation
𝑆
Singular value 𝐾 Galerkin Dynamical
𝐳 decomposition 𝜈𝑃 projection equation (ROM)
𝐳 = ෍ 𝐳෤ 𝑟 𝝂𝑟
𝑟=1 d෤𝐳𝑟 Projection-
𝐳1 𝜈1 = f(෤𝐳𝑟 )
𝐾: most energetic modes dt based ROM
𝐾≪𝑃 𝑟 = 1,2, … , 𝐾
𝐙 = 𝐳1 𝐳 2 … 𝐳 𝑆 𝝂 = [𝝂1 𝝂2 … 𝝂𝑃 ] 𝐳෤ ∈ ℝ𝐾
: linear reduced states
S: number of snapshots 𝑃: full rank modes
Snapshots Convolutional autoencoder Long short-term memory nets

Spatial
𝐳𝑆 compression Evolver
ሚ𝐡 → ℱ𝑒 𝐳 ሚ𝐡 𝐲 → ℱ𝑑 𝐳෤ Convolutional
𝐳1 AE-based
ROM
ሚ𝐡 ∈ ℝ𝐾 , 𝐾 ≪ 𝑆
Training data Non-linear latent states Latent states evolved using LSTMs
Figure 3.1: Illustration of the projection and autoencoder (AE) based

reduced-order modeling. (i) Projection-based ROM creates an orthonor-
mal basis functions (modes) for the reduced representation, which is fol-
lowed by the Galerkin projection on truncated basis functions. (ii) Con-
volutional autoencoder ROM framework is built from full-order data
using nonlinear encoding and decoding neural space without requiring
the underlying governing equations. Refer to the details of the variables
in the main text in section 3.2.
with small Kolmogorov n-width, the POD modes optimally reduce the dimension
of the nonlinear unsteady flows [16, 51]. However, in general, these empirical
projection-based reduced-order models can come at the cost of large subspace di-
mensions for turbulence or convection-dominated problems characterized by large
Kolmogorov n-width.
3.2.2 Autoencoder-based reduced-order modeling

To address the drawbacks of the projection-based ROMs, the use of autoencoders
provides a promising alternative for constructing the low-order projections of the
state vector snapshot matrix Z. In an autoencoder, F is trained to output the same
input data Z such that Z ≈ F (Z; θ AE ), where θ AE are the parameters of the end-
to-end autoencoder model. The process is to train the parameters θ AE using an
23
iterative minimization of an error function E
θ AE = argminθ AE [E(Z, F (Z; θ AE ))]. (3.9)
For the use of the autoencoder as a dimension compressor, the dimension of the
low-order space called the latent space, say H̃, is smaller than that of the input or
output data Z. When we obtain the output F (Z) similar to the input such that
Z ≈ F (Z), the latent space is a low-dimensional representation of its input which
provides a low-rank embedding. In an autoencoder, the dimension compressor is
called the encoder Fe and the counterpart is the decoder Fd . Using the encoder-
decoder architecture, the internal procedure of the autoencoder can be expressed as
H̃ = Fe (Z), Z = Fd (H̃).
For a general nonlinear system, one can construct the subspace projection as
a self-supervised DL parametrization of the autoencoders without any SVD pro-
cess of the state Z. Instead, the decomposition can be achieved by generating the
trainable layers of the encoder and decoder space such that the error function is
minimized. By using a series of convolutional and nonlinear mapping process,
the autoencoder can lead to an efficient construction of a compressed latent space
to characterize the reduced dynamics of a given nonlinear system. The autoen-
coders can be interpreted as a flexible and nonlinear generalization of POD [15].
Both projection-based POD and convolutional autoencoder help in reducing the
dimensionality of the incoming data Z by encoding dominant patterns from the
high-dimensional data. The illustration of the projection and autoencoder (AE)
based reduced-order modeling in shown in Fig 3.1. We next briefly review the
mathematical formulation of convolution and recurrent neural networks that will
be useful for the formulation of hybrid DL-ROMs.
3.3 Deep learning methods
3.3.1 Convolutional neural networks

CNNs [108] were originally developed to deal with structured data like 2D images
or 3D videos and are currently amongst the most successful neural architectures
24
CM: 1 CM: 2 CM: 3 CM: 4
FC
Input:
𝐒 𝑛 ∈ ℝ𝑁𝑥 ×𝑁𝑦 ×𝑁𝑧
Output:
…
…
…
𝐀𝑛c ∈ ℝ𝑁ℎ
…
y
x
z
3D NS flow snapshot
Figure 3.2: An abstract representation of a convolutional neural network

(CNN) architecture for reduced-order modeling. The convolution step is
denoted by blue arrows. Each neural layer outputs convolutional maps
(CM) for extracting features. These maps are further connected with
fully-connected (FC) networks to obtain the low-dimensional represen-
tation Anc from a full-order snapshot Sn . Note that Nh << (Nx ×Ny ×Nz ).
in the computer vision community. When dealing with structured data, CNNs are
a powerful alternative to fully connected networks. This is primarily accounted
to their ability to exploit local connections present in structured data, as well as
reducing the number of trainable parameters due to shared weights of its filter
bank. High correlations are common in structured data and can be extracted as
dominant features using the convolution process. Stationarity and stability to local
translations in the dataset are leveraged in CNNs [109].
A typical CNN consists of a series of convolutional layers of the form g =
CK (f), acting on a M dimensional input f(x) = ( f1 (x), . . . , fM (x)) by applying a
bank of filters K = kl,l ′ , l = 1, . . . , N, l ′ = 1, . . . , M and nonlinearity σ

!
M
gl (x) = σ ∑ fl ′ ⋆ kl,l ′ (x) . (3.10)
l ′ =1
This operation produces a N-dimensional output g(x) = (g1 (x), . . . , gN (x)) often
referred to as the feature maps or convolutional maps. The ⋆ sign represents the
standard convolutional process, which allows local features to be extracted from a
25
Euclidean space, and is given as:
Z
f x − x′ k x′ dx′ .

( f ⋆ γ)(x) = (3.11)
Ω
−1
We employ the sigmoid activation σ (z) = (1 + e−z ) function for introducing
point-wise nonlinearity. This allows nonlinear flow features such as vortex pat-
terns to be captured in the discrete convolutional process. Additionally, a pooling
or downsampling layer g = P(f) may be used, defined as
fl x′ : x′ ∈ N (x) , l = 1, . . . , N,

gl (x) = P (3.12)
where N (x) ⊂ Ω is a neighborhood around x and P is a pooling operation such as

L1 , L2 or L∞ norm. A convolutional network is constructed by composing several
convolutional and pooling layers, obtaining a generic compositional representation
UθCNN ( f ) = (CK (L) · · · P · · · ◦CK (2) ◦CK (1) ) ( f ) (3.13)
where θCNN = K (1) , . . . , K (L) is the hyper-vector of the network parameters con-

sisting of all the filter banks. The model is said to be deep if it comprises multiple
CNN layers. Notably, Eq. (3.10) is modified slightly if the convolutional blocks
are skipped on more than one element of the input function along any Cartesian
direction. The skipping
h lengths
i along the three directions of the input is termed
as the stride sL = sx sy sz and is an important hyperparameter for the dimen-
sionality reduction. CNNs possess multi-scale characteristics which allow them
to scale easily to three-dimensional Euclidean space as shown in Fig 3.2. For the
3D CNN operation of the CRAN architecture, the filter banks extract flow features
from the 3D space. The input to the convolutional layer is a 4D tensor, except for
the input layer. The Lth layer takes each 3D slice of the tensor and convolutes them
with all the kernels which create a 4D tensor.
The key advantage of CNNs is that they automatically avoid the cure of spatial
dimensionality. At the same time, the convolutional operators at each layer have
a constant number of parameters, regardless of the input size Nx × Ny × Nz . This
makes them efficient for general reduced-order modeling. For instance, the appli-
cation of CNNs as a reduced-order model has been explored by Miyanawala and
26
Jaiman [17] to predict the bluff body forces via a Euclidean distance function for
geometries. For our hybrid DL-ROM technique in this dissertation, we consider
both 2D and 3D convolutional neural networks. They are utilized to extract rel-
evant features from the unsteady flow data to construct the reduced-order states.
CNNs hence enable the extraction of the low-dimensional states from the high-
dimensional information via flexible neural networks. In the following section, we
see how can the low-dimensional states be evolved in time.
3.3.2 Long short-term memory networks

When dealing with long time series data, neural networks require persistence and
retention of information. Recurrent neural networks have been created to solve the
problem of information retention while training and making physical predictions.
Despite the success of recurrent neural networks, several stability issues have been
reported. The most common is the vanishing gradient as plane recurrent neural
networks cannot learn long-term data dependencies.
To address the issue of long-term dependency in the data, long short-term mem-
ory networks [110] are utilized. The LSTM networks are designed with the default
characteristic of retaining the information for longer periods and be able to use
them for prediction depending on its relevance and context. The LSTM manages
the progression of information through these gates by specifically including data,
eliminating or letting it through to the following cell. To understand this, let us
denote the input gate by i, output gate by o and forget gate by f. The cell state is
represented as X and the cell output is given by Y, while the cell input is denoted
as A. The equations to compute LSTM gates and states can be written as follows:
ft = σ (Wf .[Yt−1 , At ] + bf )
it = σ (Wi .[Yt−1 , At ] + bi )
X̃t = tanh(Wc .[Yt−1 , At ] + bc )
(3.14)
Xt = ft ∗ Xt−1 + it ∗ X̃t
ot = σ (Wo .[Yt−1 , At ] + bo )
Yt = ot ∗ tanh(Xt ),
27
Y1 Y2 Yi Y
X0 LSTM X1 LSTM … X i−1 LSTM LSTM
X
A1 A2 Ai A
(a)
Y𝑠 Y 𝑠+1 Y 𝑠+𝑝−1 Y 𝑠+𝑝
…
Encoder states
LSTM LSTM LSTM LSTM
Encoder states
A𝑠+1 … A𝑠+𝑝−1 A
𝑠+𝑝
A𝑠
LSTM
…
LSTM LSTM
LSTM LSTM
LSTM
… LSTM LSTM
A1 A2 …
A1 A
𝑠−1
A2A … As−1
𝑠
As
(b)
Figure 3.3: (a) Closed-loop and (b) encoder-decoder type recurrent neural
networks. 14
where W are the weights for each of the gates and X̃ is the updated cell state. σ
15
is the sigmoid activation function. These states are proliferated ahead through the
model and parameters are refreshed by backpropagation through time. The forget
gate assumes a critical part in dampening the over-fitting by not holding all data
from the past time steps. Further details can be found in [111].
In the current work, we explore two variants of LSTM-RNNs for time series
prediction of low-dimensional states
• Closed-loop recurrent neural network
• Encoder-decoder recurrent neural network
A schematic of a closed-loop and encoder-decoder recurrent neural network is de-

picted in Fig. 3.3. Closed-loop takes one single input and predicts the output and
this value is taken as input to the second prediction. In this manner, this architecture
forecasts an entire sequence of desirable length with one single input. The encoder-
decoder architectures with long short-term memory networks are widely adopted
28
for both neural machine translation (NMT) and sequence-to-sequence (seq2seq)
prediction problems. The primary advantage of the encoder-decoder architecture
is the ability to handle input and output sequences of text with variable lengths.
A single end-to-end model can also be trained directly on the source and target
sentences. It takes an input sequence of variable length and encodes the whole
information through a series of LSTM cells. There is no direct output in the en-
coding process. The output is generated via a decoder network through a series
of LSTM cells. The input and output time series can have multiple features. The
time history of a few dominant modes can be used together as multi-featured data
to train the network. In the decoding process, the state and output generated by the
previous cell are taken as input for the LSTM cell. Hence, using efficient reduced-
order modeling alongside deep learning, hybrid DL-ROMs are constructed for
data-driven predictions.
3.4 Proper orthogonal decomposition-based recurrent

neural network
This DL-ROM technique proposed by Bukka et al. [20] utilizes high-fidelity time
series of snapshot data obtained from the full-order simulations or experimen-
tal measurements. These high-dimensional data are decomposed into the spatial
modes (dominant POD basis), the mean field, and the temporal coefficients as the
reduced-order dynamics using the Galerkin projection of POD. The spatial modes
and the mean field form the offline database, while the time coefficients are prop-
agated and learned using variants of RNNs: closed-loop or encoder-decoder type
as discussed earlier. This projection and propagation technique is hence called the
POD-RNN.
Let Y = {y1 y2 ... yn } ∈ Rm×n be the snapshot matrix. Here, yi ∈ Rm is the
snapshot at time t i and n represents the number of such snapshots. m ≫ n are the
number of data probes, for instance, the number mesh nodes in the full-order sim-
ulation. The target is to predict the future states: ŷn+1 , ŷn+2 , ... using the training
dataset Y. The POD-RNN framework can be constructed by the following three
step process:
29
𝐲ത
𝐲𝑛 𝐀𝑛𝜈 ෡𝑛+1
𝐀 𝜈 𝐲ො 𝑛+1
𝐲ො 𝑛+1 ෡𝑛+1
𝐀 𝜈
෡𝑛+2
𝐀 𝜈 𝐲ො 𝑛+2
𝒱T … 𝐑𝐍𝐍 𝒱
+
…
…
…
𝐲ො 𝑛+𝑝−1 ෡𝑛+𝑝−1
𝐀 𝜈 ෡𝑛+𝑝
𝐀 𝐲ො 𝑛+𝑝
𝜈
Figure 3.4: Schematic of the POD-RNN framework with a block diagram

for the iterative prediction. With one input driver yn , the predictions
ŷn+1 , ŷn+2 , ..., ŷn+p are achieved autonomously. The diagram illustrates
the prediction of p number of time steps from ŷn+1 to ŷn+p . The dashed
lines imply an iterative process of generating all the predictions between
ŷn+2 and ŷn+p using the POD-RNN method.
Step 1: Construct the POD modes ν for the dataset Y

Given the n snapshots, determine the offline database: the temporal mean vec-
tor (y ∈ Rm ) and the POD basis modes (ν ∈ Rm×k ) using the snapshot matrix Y.
Here k < n ≪ m and k is selected based on the energy hierarchy of the POD modes.
This projection aims to reduce the order of the high-dimensional data from O(m)
to O(k). The low-dimensional states are obtained linearly using the POD recon-
struction technique
yi ≈ y + νAiν , i = 1, 2, ..., n, (3.15)
where Aiν = [ai1 i2 ik T k

ν aν ... aν ] ∈ R are the time coefficients of the k most energetic
modes for the time instant t i . The low-dimensional states Aν = {A1ν A2ν ... Anν } ∈
Rk×n are hence generated for the dataset Y. Once generated, the problem boils
n+1 n+2
down to predict Âν , Âν , ... using the reduced-order state Anν in an iterative
manner. For a detailed analysis on linear or nonlinear POD reconstruction, readers
can refer to [87, 112].
30
Step 2: Supervised learning and prediction of the low-dimensional states using
LSTM-RNN
Training consists of learning a dynamical operator gν that allows to recurrently
predict finite time series of the low-dimensional states. We employ LSTM-RNN,
which exploits both the long-term dependencies in the data and prevents the vanish-
ing gradient problem during training. This is employed as a one-to-one dynamical
mapping between the low-dimensional states Ai−1 i
ν and Aν .
i
Aiν = fν (yi ), Âν = gν (Ai−1
ν ; θν,evolver ), i = 1, 2, ..., n, (3.16)
where fν is the known function relating the low-dimensional state and the full-
order data. The algebraic relation in Eq. (3.15) can be re-written as Aiν ≈ ν T (yi −
y) ≈ fν (yi ). Hence, fν is the mapping function utilized to obtain the POD time
coefficients from a full-order snapshot. The term gν is the trainable operator
i
parametrized by θν,evolver , which outputs a time advanced state Âν with an input
Ai−1
ν . To find θν,evolver , one can set the LSTM-RNN such that the low-dimensional
states {A1ν A2ν ... An−1
ν } are mapped to the time advanced low-dimensional states
2 3 n
{Âν Âν ... Âν } in a way that gν mapping is one-to-one
i
Âν = gν (Ai−1
ν ; θν,evolver ), i = 2, 3, ..., n, (3.17)
where θν,evolver are the weight operators of the LSTM cell. Once trained, θν,evolver
forms an offline parameter recurrent space which generates a finite amount of the
predicted low-dimensional states. This closed-loop network predicts the output it-
eratively with the same trained weights, meaning that the generated output is fed
as an input for the next prediction.
Step 3: Reconstruction to the point cloud state ŷn+1

In Step 2, the reduced-order dynamics is predicted using the LSTM-RNN.
n+1
Once Âν is generated, it is straightforward to reconstruct the full-order state at
t n+1 as follows:
n+1
ŷn+1 ≈ y + ν Âν . (3.18)
Hence, the predicted low-dimensional states are linearly combined with the tem-
31
poral mean vector (y ∈ Rm ) and the POD basis modes (ν ∈ Rm×k ). An illustration
of the entire predictive process is shown in Fig. 3.4. This technique of the POD-
RNN can be capable of predicting the FSI motion or unsteady flow fields for finite
time steps ŷn+1 , ŷn+2 , ..., ŷn+p autonomously provided that the error is within the
acceptable range of accuracy.
3.4.1 Dataset preparation, training and prediction

In the POD-RNN, we train and predict the time coefficients obtained from the
POD analysis of the full-order dataset. Let Aν = {A1ν A2ν ... Anν } ∈ Rk×n be the
low-dimensional states obtained from the POD analysis of the snapshot matrix Y.
The scaling of Aν is conducted as
Aiν − Aν,min
Aiν,s = , i = 1, 2, ..., n, (3.19)
Aν,max − Aν,min
where Aν,max and Aν,min are the maximum and minimum values in Aν , respec-
tively, and arranged as a vector. This ensures Aiν,s ∈ [0, 1]k . The obtained dataset
matrix Aν,s = A1ν,s A2ν,s . . . Anν,s ∈ Rk×n consists of the normalised time coeffi-

cients.
The training consists of finding the evolver parameters of the LSTM-RNN

θν,evolver such that the observable loss is minimized
"
i i−1 2#
1 n Aν,s − gν (Aν,s ; θν,evolver ) 2
θν,evolver = argmin(θν,evolver ) ∑ 2
. (3.20)
n − 1 i=2 Aiν,s 2
This loss function is iteratively minimised over the entire training sequence using
the adaptive moment optimization (ADAM) [113].
3.5 Convolutional recurrent autoencoder network

The POD-RNN method can provide an optimal low-dimensional space (encoding)
in which the modes are not only computationally inexpensive to obtain but can be
physically interpretable as well [87]. However, there are several problems with
32
the POD encoding for highly nonlinear flow physics: (a) the encoding scales lin-
early, which may cause significant loss of the flow physics characterized by a large
Kolmogorov n-width, (b) the POD reconstruction of the flow problems dominated
by turbulence often results in the slow decay of the POD energy spectrum, imply-
ing that the number of POD modes can increase significantly and be difficult to
learn, and (c) the POD basis can have additional orthogonality constraint on the
low-dimensional space which can limit its flexibility in general. So an alternative
approach is to utilize a much more flexible encoding that can address the above
challenges. One method is to use CNNs instead of POD, which leads to a non-
intrusive type of DL-ROM framework called the CNN-RNN or simply the CRAN.
While operating synchronously in a nonlinear neural space, this projection and
propagation technique extracts the low-dimensional embedding whereby the flow
variables are extracted via CNNs and the encoding evolved via LSTM-RNN. Since
there is no knowledge of the mean or basis vectors here, we construct a decod-
ing space of transpose convolution that up-samples the low-dimensional encoding
back to the high-dimensional space. It is assumed that the solution space of the
unsteady flow attracts a low-dimensional subspace, which allows the embedding in
the high-dimensional space. This end-to-end convolutional autoencoder architec-
ture is illustrated in Fig. 3.5.
The CRAN framework needs a high-fidelity series of snapshots of the flow
field obtained by full-order simulations. Let S = {S1 S2 ... Sn } ∈ RNx × Ny ×n denote
the 2D snapshots of a field dataset (such as pressure or velocity data from a flow
solver), where Si ∈ RNx ×Ny is the field snapshot at time t i and n represents the num-
ber of such snapshots. Nx and Ny are the number of data probes in the respective
Cartesian axes. For instance, here, the number of fixed (Eulerian) query points
introduced in the moving point cloud. These probes are structured as a spatially
uniform space for the CRAN architecture as it relies on the field uniformity. These
probes are optimally selected based on the field convergence and the interface load
recovery using the snapshot-FTLR method as discussed in section 3.6. The target
of the CRAN-based end-to-end learning is to encode-propagate-decode the future
n+1 n+2
values at the field probes: Ŝ , Ŝ , ... using the training dataset S. The CRAN
framework is constructed using the following process:
33
Spatial down-sampling
… Reshape
Conv2D #1 Conv2D #2 Conv2D #𝑛𝑐𝑜𝑛𝑣

Input:
𝐒𝑛 FC # 𝑛𝑓𝑢𝑙𝑙 ෡𝑖+1
LSTM-RNN 𝐀 𝑐
Generative output: Temporal Temporal

𝐀𝑛𝑐 ෡ 𝑐 𝑛+1
𝐀
𝐒෠ 𝑛+1 , … , 𝐒෠ 𝑛+𝑝 encoding decoding
𝐀𝑖𝑐
෡ 𝑐 𝑛+𝑝
𝐀
DeConv2D #𝑛𝑐𝑜𝑛𝑣 DeConv2D #(𝑛𝑐𝑜𝑛𝑣 −1) DeConv2D #1
… Transform FC # 𝑛𝑓𝑢𝑙𝑙
Spatial prolongation
6
Figure 3.5: Schematic of the CRAN framework for flow field prediction. The
encoding is achieved by reducing the input dimension from Nx × Ny to
Ac via CNNs (Conv2D) and the fully connected networks. The decod-
ing is achieved using the fully connected layers and the transpose CNNs
(DeConv2D). Between the encoding and decoding space, the LSTM-
RNN evolves the low-dimensional state Ac .
Step 1: Find the nonlinear encoding feature space for the dataset S
Given the n snapshots of any field data, construct a trainable neural encoder space
using layers of linear convolutional kernels with nonlinear activation (Conv2D) as
shown in Fig. 3.5. The dimensionality of the field 2D probes is conveniently and
gradually down-sampled using the convolutional filters and the plain feed-forward
networks until a finite size of the low-dimensional state is reached, similar to the
POD encoding as discussed earlier. The update of time coefficient can be expressed
as
Aic = fc (Si ; θc,enc ), i = 1, 2, ..., n, (3.21)
where fc is the trainable encoder space that is parametrized by θc,enc . The time
coefficients Aic = [ai1 i2 ih T h i
c ac ... ac ] ∈ R represent h encoded features at time t . The
low-dimensional states Ac = {A1c A2c ... Anc } ∈ Rh×n of the dataset S are determined
in a self-supervised fashion of the autoencoder, with no energy or orthogonality
34
constraint unlike the POD. Here, h < n ≪ (Nx × Ny ) and h is the unknown hy-
perparamter for the optimal feature extraction based on the input dataset S. This
projection reduces the order of high-dimensional field from O(Nx × Ny ) to O(h).
n+1 n+2
After the decomposition, the problem boils down to predict Âc , Âc , ... using
n+1 n+2
the low-dimensional state Anc and spatially up-sample to Ŝ , Ŝ , ... as genera-
tive outputs.
Step 2: Supervised learning and prediction of the low-dimensional states using

LSTM-RNN
Similar to the POD-RNN, the training of the low-dimensional space consists
of learning a one-to-one dynamical operator gc that allows to predict a certain se-
ries of the low-dimensional states. Without a loss of generality, we employ the
LSTM-RNN in a closed-loop fashion. The LSTM-RNN is employed as a state
transformation between the low-dimensional states Aci−1 and Aic . Instantiate the
network such that the low-dimensional states {A1c A2c ... An−1
c } are mapped to the
2 3 n
time advanced low-dimensional states {Âc Âc ... Âc } so that dynamical operator
gc is a one-to-one transformation
i
Âc = gc (Ai−1
c ; θc,evolver ), i = 2, ..., n, (3.22)
where θc,evolver are the weight operators of the LSTM cell. Likewise, once trained,
θc,evolver also forms an offline recurrent parameter space which predicts finite time
steps of the self-supervised features obtained from the neural encoder space.
n+1
Step 3: Generating the high-dimensional time advanced state Ŝ
n+1
Once Âc is generated, there is a need to prolongate the time advanced fea-
tures at t n+1 using gradual reverse convolution (DeConv2D) as these features are
often hard to interpret. As there is no knowledge of the mean or the POD basis vec-
tors, we rely on decoding these low-dimensional features gradually using a decoder
space which is a mirror of the encoder space.
n+1 n+1
Ŝ = fc−1 (Âc ; θc,dec ), (3.23)
35
where fc−1 is the trainable decoder space that is parametrized by θc,dec . The trained
LSTM-RNN is employed to iteratively generate the desired future states
n+1 n+2 n+p
Ŝ , Ŝ ...., Ŝ starting from Sn . This technique of the CRAN on the flow
variables is self-supervised and is capable of predicting the flow variables at the
fixed probes provided that the prediction error is within the acceptable range of
accuracy. The complete predictive process is illustrated in Fig. 3.5.
3.5.1 Dataset preparation, training and prediction

We normalise the flow dataset snapshot matrix S = {S1 S2 ... Sn } ∈ RNx × Ny ×n by
first finding the fluctuation around the temporal mean using
S′i = Si − S, i = 1, 2, ..., n, (3.24)
where S = 1n ∑ni=1 Si is the temporal average over the entire dataset and
S′ = {S′1 S′2 ... S′n } ∈ RNx × Ny ×n are the fluctuations around this mean. Next the
scaling of S′ is conducted as
S′i − S′min
S′is = , i = 1, 2, ..., n, (3.25)
S′max − S′min
where S′max and S′min are the maximum and the minimum vales in S′ , respectively,
and arranged as a matrix. The obtained dataset matrix S′s = S′1 ′2 ′n is fur-

s Ss . . . Ss
ther broken up into a set of Ns finite time training sequences, where each training
sequence consists of Nt snapshots. The training dataset with the above modifica-
tions has the following form:
S = Ss′1 Ss′2 . . . Ss′Ns ∈ [0, 1]Nx ×Ny ×Nt ×Ns

(3.26)
h i
where each training sample Ss′ j = S′1 S ′2 . . . SNt is a matrix consisting of the
s, j s, j s, j
scaled database.
A loss function is constructed that equally weights the error in the full-state
reconstruction and the evolution of the low-dimensional representations. The tar-
h parametersiθc = {θc,enc , θc,dec , θc,evolver }
get of the training is to find the CRAN
′j
such that for any sequence Ss = S′1 ′2 Nt
s, j Ss, j . . . Ss, j and its corresponding low-
36
h i
dimensional representation A1c, j A2c, j . . . ANc,tj the following error between the truth
and the prediction is minimized:
2
β Nt S′is, j − fc−1 ( fc (S′is, j ; θc,enc ); θc,dec )
2
θc = argminθc ( ∑ 2
Nt i=1 S′is, j
2
2
(3.27)
(1 − β ) Nt Aic, j − gc (Aic, j ; θc,evolver )
2
+ ∑ 2
)
Nt − 1 i=2
Aic, j
2
where β = 0.5. The value of β is chosen in order to give equal weightage to the
errors in the full state reconstruction and feature predictions, respectively. This
loss function is iteratively minimised for all the j = 1, 2, . . . , Ns training sequences
using the ADAM [113].
3.6 Snapshot-field transfer and load recovery

We introduce a novel procedure for the partitioned mesh-to-mesh field transfer
between the full-order grid and the neural network grid. A convolutional neural
network relies on a spatially uniform input data stream for identifying patterns in
datasets. This is accounted for the uniform dimension and operation of the adap-
tive CNN kernels. Given that most practical FSI problems are modeled in a highly
unstructured and body conformal mesh, there is a need to develop a flow field data
processing step to interpolate the scattered information as snapshot images before
learning them in the CRAN framework. Consequently, in this section, we intro-
duce a general data processing step that can be utilized for mapping the flow field
variables from an unstructured grid of the full-order model to a uniform grid of
the DL-ROM. We achieve this via interpolation and projection of the field infor-
mation in an iterative process that allows a recoverable interface force data loss.
Once this loss is observed in the training forces, we correct them by reconstructing
to a higher-order CFD force. We select the interface force as the primary criteria
because the boundary layer forces are highly grid sensitive. This iterative process
is cyclically performed as shown in Fig. 3.6. We refer to the entire cyclic process
37
(3) Pixelated
ത𝑏 (𝑡)
force: 𝐅
(2) Level-set (4) Damp noises:

ζ𝐅ത𝑏 (𝑡)
𝚽
DL grid
𝐒(X, Y, 𝑡)
(1) Field transfer (5) Interface load
recovery: Ψ𝐅 ത𝑏 (𝑡)
CFD grid
𝐬(x 𝑡 , y 𝑡 , 𝑡)
Figure 3.6: Illustration of an iterative cycle for mesh-to-mesh field transfer

and load recovery to determine the CRAN snapshot grid. See the details
of all variables in the main text.
as a snapshot-field transfer and load recovery (snapshot-FTLR) method.

Herein, our intent is to find the best DL-ROM grid that can recover the full-
order interface load correctly and capture the Lagrangian-Eulerian interface. While
the loss of data information is observed in the training data, it is assumed that
the loss is unchanged and can be used as correction during the predictions. The
proposed snapshot-FTLR technique can be divided into five key steps, which are
as follows:
1. Field transfer: The first step involves the field transfer from a highly unstruc-
tured moving point cloud (x(t), y(t)) to a reference uniform grid (X, Y). The
size of the grid can be chosen Nx × Ny . We use Scipy’s griddata function
[114] to interpolate the scattered CFD data and fit a surface. This function
generates the interpolant using Clough-Tocher scheme [115] by triangulat-
ing the scattered data s(x(t), y(t),t) with Quickhull algorithm and forming a
piecewise cubic Bezier interpolating polynomial on each triangle. The gra-
dients of the interpolant are chosen such that the curvature of the interpolated
38
surface is approximately minimized [116, 117]. After achieving this mini-
mization, the field values at the static Eulerian probes Nx × Ny are generated
as S(X, Y,t).
2. Level-set: With the aid of learned interface description, a level Φ is assigned

for all the cells on the uniform grid. The exact interface description is known
for a stationary solid boundary, which is then used for synchronous struc-
tural motion prediction via the POD-RNN framework. The process is fur-
ther demonstrated in section 3.4. This step provides the identification of the
solid, fluid and interface cells on the DL-ROM grid.
3. Pixelated force: The next step is the calculation of the forces exerted by
the interface cells that contain the rigid body description. We refer to these
forces as pixelated forces. The method of measuring the pixelated force in-
corporates the construction of the Cauchy stress tensor in the interface pixels.
As shown in Fig. 3.7, for an interface cell k at time instant n, the pixelated
n
force fk for a Newtonian fluid can be written as
n n n n n
fk = (σa;k − σb;k ).nx ∆y + (σc;k − σd;k ).ny ∆x, (3.28)
n is the Cauchy stress tensor for any point inside the cell k. As
where σ∗;k
depicted in Fig. 3.7, we calculate this tensor at the mid-points of cell faces
a−d−b−c via finite difference approximation. nx and ny are normals in the
x and y-direction, respectively, whereby ∆x and ∆y denote the cell sizes. For
the present case, we only consider the pressure component while calculating
n
fk .
To obtain the total pixelated force signal, these individual pixelated forces
are summed over all the interface cells as
NF
n n
Fb = ∑ fk . (3.29)
k=1
where NF denotes the number of interface cells. These total pixelated force
1 2 n
signals Fb = {Fb Fb ... Fb }, however, can be corrupted with missing or noisy
information compared to its higher-order counterpart. This is accounted due
39
Fluid cells
Δy (𝑖, 𝑗 + 1)
Interface cells
Δx
1 c 2 Solid cells
(𝑖 − 1, 𝑗) b (𝑖, 𝑗) a (𝑖 + 1, 𝑗)
Face mid-point
4 d 3
𝐒≠0
(𝑖, 𝑗 − 1)
𝐒=0
Figure 3.7: Identification of interface cells in the DL-ROM grid and finite
difference interpolation of field at the cell faces (green cross). The blue
nodes contain the flow field values and the red dots are marked as zero.
to the loss of sub-grid information that leads to a coarser calculation of forces

on the sharp interface. Thus, these output signals need to be corrected.
4. Damp noises: To correct the pixelated signals, we first apply a moving aver-
age Gaussian filter ζ to smoothen the signals. This is achieved by damping
the high-frequency noises in these signals that are encountered due to a full-
order interface movement on a low-resolution uniform grid. This damping
smoothens the force propagation for easier reconstruction. To smooth a pix-
n
elated force signal Fb at a time step n, we perform the following operations:
(a) collect the time series high-frequency noisy signals Fb of window length
2f +1
n− f n−1 n n+1 n+ f
Fb = {Fb ... Fb Fb Fb ... Fb }, (3.30)
(b) select Gaussian-like weights of the specified window length with mean
n
weight wn for Fb
n+ f
w = {wn− f ... wn−1 wn wn+1 ... wn+ f }, with ∑ wi = 1 (3.31)
i=n− f
40
n
(c) apply the weighted average filter to damp the noise at Fb
n+ f
n i n
Fb := ∑ wi Fb = ζ Fb . (3.32)
i=n− f
5. Interface force recovery: The smooth pixelated force propagation ζ Fb can

still lack the mean and derivative effects compared to its full-order counter-
part FΓfs . This is where we introduce the functional mapping Ψ as described
in Algorithm 1. If this mapping can recover the bulk forces correctly, then
we can utilize the selected grid. If not, then refine the grid and repeat the
above steps from (1)-(5), until we find the best snapshot grid that recovers
these bulk quantities with acceptable accuracy.
Algorithm 1: Functional reconstruction mapping Ψ for a higher-order force recovery

on the DL-ROM grid.
Algorithm 1
Input: De-noised pixelated force signals Fb , full-order FSI forces FΓfs

Output: Reconstructed pixelated force signal ΨFb
To reconstruct smooth pixelated force signals Fb :
1. Collect n time steps of denoised pixelated force and full-order FSI forces:
1 2 n−1 n
Fb = {Fb Fb ... Fb Fb };
1 2 n−1 n
FΓfs = {FΓfs FΓfs ... FΓfs FΓfs };
2. Get the mean and fluctuating force components:

′
Fb = Fb − mean(Fb );
′
FΓfs = FΓfs − mean(FΓfs );
3. Define the time-dependent derivative error using E c :

′ ′ ′ ′
E c = (FΓfs − Fb )./(Fb ) with Fb ̸= 0;
4. Reconstruct the smooth pixelated force signals with mean and derivative cor-
rections:
′ ′
Fb := Fb + mean(FΓfs ) + mean(E c )Fb = ΨFb ;
41
Algorithm 2: Iterative snapshot DL-ROM grid search via interface force recovery.
Algorithm 2
Input: Full-order field data s = {s1 s2 ... sn } and FSI forces

1 2 n
FΓfs = {FΓfs FΓfs ... FΓfs }
Output: Grid selection Nx × Ny
Initialise a uniform grid (Nx × Ny ) of cell size ∆x = ∆y:

while:
1. Project point cloud CFD data s(x(t), y(t)) ∈ Rm×n on snapshot grid S(X, Y):
S ← s s.t. S ∈ RNx ×Ny ×n ;
2. Apply learned level-set function Φ ∈ RNx ×Ny ×n on S(X, Y) element wise:

S := Φ ∗ S;
3. Calculate pixelated force signals Fb from interface cells Γfs in S using Eqs.
(3.28) - (3.29);
4. Damp the high-frequency noises in pixelated signals Fb via moving average

Gaussian filter ζ :
Fb := ζ Fb using Eqs. (3.30)-(3.32);
5. Reconstruct the smooth pixelated force Fb to full-order by functional mapping

Ψ:
Fb :≈ ΨFb using Algorithm 1;
6. if ((∥Fb − FΓfs ∥/∥FΓfs ∥) ≤ ε ):

break;
else:
Refine the grid Nx := 2Nx , Ny := 2Ny ;
Remark 1. The mesh-to-mesh field transfer and load recovery for the snapshot
DL-ROM grid search are expressed in a similar procedure as Algorithm 2. Once
selected, the chosen uniform grid is utilized for learning the flow variables in the
CRAN framework with the POD-RNN operating on the moving interface. Algo-
rithm 3 essentially employs the aforementioned interface load recovery to extract
the higher-order forces from the synchronized field and interface predictions.
42
Algorithm 3: Extracting FSI forces via predicted fields on the DL-ROM grid and the ALE
displacement.
Algorithm 3
n+1 n+2 n+p

Input: Field predictions from CRAN Ŝ = {Ŝ Ŝ ... Ŝ }, displacement pre-
n+1 n+2 n+p
dictions from POD-RNN ŷ = {ŷ ŷ ... ŷ }, calculated interface mapping Ψ
from training data
n+1 n+2 n+p
Output: Full-order force predictions: Fˆ fs = {Fˆ fs Fˆ fs ... Fˆ fs }
Γ Γ Γ Γ
To extract full-order FSI forces:
1. Apply predicted level-set function Φ̂ ∈ RNx ×Ny ×p from POD-RNN on Ŝ ∈

RNx ×Ny ×p element wise:
Ŝ := Φ ∗ Ŝ;
2. Calculate pixelated force signals Fˆ b from interface cells Γfs in Ŝ using Eqs.
(3.28) - (3.29);
3. Damp high-frequency noises in extracted pixelated signals Fˆ b via moving av-

erage gaussian filter ζ :
Fˆ b := ζ Fˆ b using Eqs. (3.30)-(3.32);
4. Reconstruct the extracted smooth pixelated force Fˆ b to full-order by known

functional mapping Ψ:
Fˆ Γfs ≈ ΨFˆ b
43
Chapter 4
Low-dimensional Analysis via

Deep Learning
The development of reduced-order models attempts to replace the high-dimensional

(i.e., low-fidelity) solution from the full-order model to a low-dimensional (i.e.,
low-fidelity) solution manifold for computational efficiency. To accomplish this,
linear projection methods such as POD or nonlinear autoencoder techniques based
on CNNs are a general choice as discussed in the previous chapter. In this chapter,
we employ both POD and CNN to obtain a low-dimensional representation from a
high-dimensional Navier-Stokes solver. These low-dimensional features are prop-
agated in time with RNNs to predict the dynamics. We investigate the data-driven
analysis of two hybrid deep learning-based reduced-order models (a) POD-RNN
and (b) CRAN for the flow past static bluff bodies. For this purpose, we assess
the unsteady flow predictions for a configuration of side-by-side cylinders with the
wake interference at a low Re. Albeit at a low Re, a side-by-side system has a
fundamental value due to the richness of nonlinear flow physics associated with
the near-wake dynamics and the vortex-to-vortex interactions. The biased gap flow
between a stationary side-by-side the cylinder is bi-stable and switches intermit-
tently through a strongly bimodal distribution associated with spontaneous sym-
metry breaking. Of particular interest is to benchmark hybrid DL-ROMs in terms
of computational efficiency and accuracy for complex (side-by-side cylinders) flow
patterns.
44
4.1 Flow past side-by-side cylinders
The canonical problem of flow past side-by-side cylinders can act as a represen-
tative test case for many multi-body systems. The dynamics associated with this
problem is still a popular topic of research among the fluid mechanics community.
The flow dynamics are highly sensitive and are directly related to the interaction
dynamics of the downstream coherent vortex patterns with the gap flow between
the cylinders. Hence, as a benchmark test case, we apply both the POD-RNN and
CRAN models for such a bi-stable problem to test the capabilities of the predic-
tion tools. This includes the prediction of the flow field and the integrated pressure
force coefficient on the cylinders.
The objective here is to sufficiently learn the phenomenon of vortex shedding
for flow past side-by-side cylinders on a given time set (training) and extrapolate
the behavior based on learned parameters (prediction). However, flow past side-
by-side cylinders is known to be a more complex (chaotic) problem with a bi-stable
solution, flip-flopping regime and gap flow which is discussed in detail by Liu et al.
[118]. The idea is to study the performance of this problem for both types of hybrid
DL-ROMs. The schematic of the problem set-up is given in Fig. 4.1 (a), where
all the information about the domain boundaries and the boundary conditions are
clearly outlined. The final mesh which is obtained after following standard rules
of mesh convergence is presented in Fig. 4.1 (b) and (c) which contains a total of
49762 triangular elements with 25034 nodes. The numerical simulation is carried
out via finite element Navier-Stokes solver to generate the train and test data. The
Reynolds number of the problem is set at Re = 100. The full-order simulation is
carried out for a total of 950 tU∞ /D with a time step of 0.25 tU∞ /D. A total of 3800
snapshots of the simulation are collected at every 0.25 tU∞ /D for the pressure field
and the x-velocity. Of those 3800 snapshots, 3200 (from 301 to 3500 steps) are
used for training and 300 (from 3501 to 3800 steps) are kept as testing. The total
train and test snapshots are N = 3500.
45
(a)
(b) (c)
Figure 4.1: The flow past side-by-side cylinders: (a) Schematic of the prob-
lem set-up, (b) full-domain computational mesh view and (c) close-up
computational mesh view.
In the sections that follow, we systematically assess the prediction of the flow
field and pressure force coefficients using the POD-RNN and CRAN for flow past
side-by-side cylinders. A discussion highlighting the significant differences in both
the models is provided for a comparative study in section 4.1.3.
46
4.1.1 POD-based recurrent neural networks
The POD-RNN model is applied on the flow past side-by-side cylinders in this
section. The POD algorithm is applied on all the train and test time snapshots
for field S = S1 S2 . . . SN ∈ Rm×N (here, m = 25034 and N = 3500) to get the

reduced order dynamics of the dataset matrix as detailed in section 3.4.
(a) (b)
(c) (d)
Figure 4.2: The flow past side-by-side cylinders: Cumulative and percentage
of modal energies. (a)-(b) for pressure field, (c)-(d) for x-velocity field.
Via POD spatial decomposition, we get the mean field S̄ ∈ Rm , the fluctuation
′
matrix S ∈ Rm×N and the N spatially invariant POD modes ν ∈ Rm×N associated
47
(a) (b)
(c) (d)
Figure 4.3: The flow past side-by-side cylinders: First four most energetic
time-invariant spatial POD modes along with % of total energy for pres-
sure field.
′ ′
with the dynamics. The eigen values ΛN×N of the covariance matrix S T S ∈ RN×N
represent the energy of POD modes. The POD energy spectrum for both pressure
and x-velocity fields is plotted in Fig. 4.2 with respect to the number of modes.
From the spectrum, it can be interpreted that around 95% of the system energy is
concentrated in first 21 modes for the pressure and first 19 for the x-velocity.
Subsequently, the reduced order system dynamics can be constructed by se-
lecting first few energetic POD modes in the analysis. Here, we select k = 25
modes for both pressure and x-velocity as it accounts for nearly 96% of the sys-
48
(a) (b)
(c) (d)
Figure 4.4: The flow past side-by-side cylinders: First four most energetic
time-invariant spatial POD modes along with % of total energy for x-
velocity field.
tem energy. The N POD modes are now simply approximated with the selected
k modes (k ≪ N), reducing the POD system to ν ∈ Rm×k . For instance, the first
four spatial POD modes for pressure and x-velocity fields along with % total en-
ergy are depicted in Figs. 4.3 and 4.4. The temporal variations of these k modes
′ ′
are obtained by Aν = ν T S . Here, Aν ∈ Rk×N , ν ∈ Rm×k and S ∈ Rm×N . These
temporal modes are conveniently divided into training (ntr steps) and testing part
(nts = N − ntr steps). Here, ntr = 3200 and nts = 300. The temporal coefficients
from 301 − 3500 time steps are considered for training and 3501 − 3800 are kept
49
(a)
(b)
Figure 4.5: The flow past side-by-side cylinders: Time histories of first eight
modal coefficients shown from 75 to 325 tU∞ /D: (a) Pressure field and
(b) x-velocity field.
50
Parameter Value
Input features k = 25
Output features k = 25
Hidden dimension 512
RNN cell LSTM
Time steps per batch ni = no = 25
Number of mini-batches ns = 50 (random)
Initial learning rate 0.0005
L-2 regularization factor 0.003
Iterations 15000 epochs
Optimizer ADAM
Table 4.1: The flow past side-by-side cylinders: Network and parameter de-
tails for the encoder-decoder type recurrent network.
for testing. For illustration, the time history of first eight POD modes for the pres-
sure and x-velocity fields is depicted in Fig. 4.5. Notably, the time prorogation
of the associated reduced-order states Aν are chaotic and represent no repeating
amplitude or frequency in this case.
The temporal behavior of these modal coefficients is learned via an encoder-
decoder type recurrent neural network (individually for pressure and x-velocity)
as it was found to work better compared to a closed-loop type (see section 3.3.2).
Before proceeding with training, the modal coefficients are normalized between 0
to 1 to prevent vanishing/exploding gradients. The primary feature of this recurrent
architecture is to encode a finite sequence of time steps (of length, say, ni ) and
decode it to predict the next set of time steps (of length no ). For example, time
steps 1 to 25 can be encoded to predict 26 until 30 steps and so on. The training
data consists of finite input-output time sequences of k time coefficients generated
in random mini-batches of size ns in every epoch (here, ns = 50, ni = no = 25,
k = 25). The parametric details of the recurrent network for pressure and x-velocity
are further outlined in Table 4.1. The test time steps of length 300 are organized
into six equal sets of input-output pairs. The size of the input ni and output no time
window is 25 here.
The result of the temporal modal predictions from the encoder-decoder type
51
(a)
(b)
Figure 4.6: The flow past side-by-side cylinders: Temporal evolution (pre-
diction) of modal coefficients from 875 till 950 tU∞ /D for (a) pressure
field and (b) x-velocity field.
52
(a) (b)
Figure 4.7: The flow past side-by-side cylinders: RMSE for the predicted
against true modal coefficients for (a) pressure (b) x-velocity.
recurrent network is plotted in Fig. 4.6 for (a) pressure and (b) x-velocity over the
test time steps from 875 − 950 tU∞ /D. For visualisation convenience, only first 8
modal coefficients are shown, although the network predicts the temporal trend in
all k = 25 low-dimensional states within a reasonable accuracy. The network input,
prediction and true values are depicted by green, red and black lines respectively
in Fig. 4.6. In spite of the drastic differences between the POD modal coefficients,
one trained encoder-decoder network is suffice to captureq
such a chaotic behavior
n n
∑n=T
n=1 (Aν −Âν )
2
for finite time steps. The root mean square error RMSE = T is plotted
n n
in Fig. 4.7 for all the predicted 25 modal coefficients. Here, Aν and Âν denote the
true and predicted modal coefficients, respectively. T is the total length of the
output time sequences (here, T = 6no = 150). The temporal prediction errors of
the low-dimensional states are in the order of 10−1 for both the fields.
n
The predicted modal coefficient Âν ∈ Rk can simply be reconstructed back
n
to the high-dimensional state Ŝ ∈ Rm using the mean field S̄ ∈ Rm and k spatial
n n
POD modes ν ∈ Rm×k as Ŝ ≈ S̄ + ν Âν . Figs. 4.8 and 4.9 depict the compari-
son of predicted and true values for P and U, respectively at test time steps 3592
(898 tU∞ /D), 3692 (923 tU∞ /D) and 3792 (948 tU∞ /D). The normalized recon-
struction error En is constructed by taking the absolute value of the differences
53
(a)
(b)
(c)
Figure 4.8: The flow past side-by-side cylinders: Comparison of predicted

and true fields (POD-RNN model) along with normalized reconstruction
error En at (a) tU∞ /D = 898, (b) tU∞ /D = 923, (c) tU∞ /D = 948 for
pressure field.
54
(a)
(b)
(c)

and true fields (POD-RNN model) along with normalized reconstruction
error En at (a) tU∞ /D = 898, (b) tU∞ /D = 923, (c) tU∞ /D = 948 for x-
velocity field.
55
(a) (b)
(c) (d)
Figure 4.10: The flow past side-by-side cylinders: Predicted and actual
(POD-RNN model) pressure force coefficients. (a) Drag (Cylinder 1),
(b) lift (Cylinder 1), (c) drag (Cylinder 2), (d) lift (Cylinder 2).
n
between the true Sn and predicted Ŝ field at any time step n and normalizing it
with L2 norm of the true data and is given by
n
|Sn − Ŝ |
En = . (4.1)
∥Sn ∥2
The high-dimensional field prediction on all the CFD nodal points allows to
56
carry the reduced numerical integration directly over the fluid-solid boundary on
the cylinders. The Cauchy stress tensor σ f is constructed with the pressure field
data and is integrated over the fluid-solid boundary to get CD,p and CL,p . Fig. 4.10
depicts the predicted and actual pressure coefficient from time steps 3501 − 3800
for the upper (Cylinder 1) and lower (Cylinder 2) cylinders. Note that the reference
length scale is the diameter of cylinder (D = 1) and the reference velocity is the
inlet velocity (U∞ =1). t denotes the actual simulation time carried at a time step
0.25 tU∞ /D. The force predictions are in good agreement with the test data.
4.1.2 CNN-based recurrent neural networks

The end-to-end nonlinear model order reduction tool based on the CRAN is now
applied to the same problem of flow past side-by-side cylinders. Same set of train-
ing (time steps 301 − 3500) and testing (time steps 3501 − 3800) data are selected
for this application.
We use the snapshot-FTLR (see section 3.6) to bring field uniformity in the DL
space, while recovering forces. A DL space of dimension 8D × 8D is selected with
≈ 5D length kept for downstream wake. Point cloud CFD training/testing data (for
instance pressure field) is interpolated and projected as spatially uniform 2D snap-
shots S = {S1 S2 ... SN } ∈ RNx ×Ny ×N in the chosen DL space. Here, Nx =Ny = 64
and N = 3500. The field uniformity reduces the model-order fidelity and unstruc-
tured mesh complexity by mapping the m-dimensional unstructured dataset on a
2D reference grid. With the chosen DL grid Nx = Ny = 64, the CRAN is employed
for the coarse-grain flow field predictions. These N 2D snapshots are divided into
training ntr = 3200 and testing nts = N − ntr = 300 parts. Following this, standard
principles of data normalization and batch-wise arrangement are adopted to gen-
erate the scaled featured input S = Ss′1 Ss′2 . . . Ss′Ns ∈ [0, 1]Nx ×Ny ×Nt ×Ns with

Ns = 128 and Nt = 25. The data preparation is detailed in section 3.5.

Convolutional recurrent autoencoder networks are trained using the dataset de-
scribed above and the low-dimensional evolver LSTM states Ac of different sizes.
Predictions are experimented with different sizes NA = Nh = 2, 4, 8, 16, 32, 64, 128,
256 for both the pressure and x-velocity training. All the CRAN models are trained
on a single Intel E5-2690v3 (2.60GHz, 12 cores) CPU node for Ntrain = 500, 000
57
(a) (b)
Figure 4.11: The flow past side-by-side cylinders: Mean normalized squared
error Ef for first Nt predicted time steps with respect to size of low-
dimensional state Nh of the CRAN for (a) pressure field and (b) x-
velocity field. Note that NA = Nh .
iterations. We maintain a mini-batch of size ns = 5 for every iteration. The objec-

tive function given by Eq. (3.27) was observed to reach a nearly steady value of
approximately 10−5 at the end of iterations. It should be noted that we are employ-
ing a closed-loop recurrent neural network in the CRAN model for this problem
i.e., the prediction from previous step is used as an input for the next prediction.
The closed-loop recurrent net uses prediction from the previous time step to
generate a new prediction. Due to the bistable nature of the problem, the errors
are compounded at every time step, thus, causing prediction trajectory to deviate
after a few finite time steps (≈ Nt = 25 time steps). To address this issue, ground
truth data is fed after every Nt predicted time steps as a demonstrator for the net to
follow the actual trajectory and avoid divergence. We follow the 1-to-Nt rule for
the prediction. This means that any ground truth snapshot predicts Nt time steps
from a trained model. Thus, in testing, we input ≈ nts /Nt ground truth data to
predict nts time steps. For instance, time step 3500 predicts sequence 3501 − 3525
steps, time step 3525 predicts 3526 − 3550 and so on.
58
(a) (b)
(c) (d)
Figure 4.12: The flow past side-by-side cylinders: Comparison of the

mapped pixel force and full-order force coefficients on the training
data due to pressure for Cylinder 1 ((a)-(b)) and Cylinder 2 ((c)-(d))
using snapshot-FTLR Ψ mapping.
Fig. 4.11 (a) and (b) depict the normalised mean squared error Ef for the first
set of Nt predicted time steps for pressure and x-velocity, respectively, with respect
to the size of low-dimensional features Nh .
n
∥Sn −Ŝ ∥22
∑n ∥Sn ∥22 +ε
Ef = , (4.2)
Nt
59
where n denotes a time instance from 3501 to 3525 time steps and ∥ . . . ∥2 denotes
n
the L2 norm over the spatial dimension. Sn and Ŝ are respectively the true and
predicted fields. The tuning of the hyperparameter Nh reveals the existence of an
optimal value of Nh for the field dataset. Referring to Fig. 4.11, the variation of
Ef vs Nh follows a nearly convex shaped optimisation curve with the minimum at
Nh = 32 for pressure and Nh = 16 for x-velocity. Based on this, we utilise Nh = 32
and Nh = 16 trained CRAN models for pressure and x-velocity, respectively, to
predict all the nts = 300 test time steps.
Figs. 4.13 and 4.14 depict the comparison of predicted and the true values for
the pressure and x-velocity fields, respectively, at time steps 3592 (898 tU∞ /D),
3692 (923 tU∞ /D) and 3792 (948 tU∞ /D). The normalized reconstruction error
En is constructed by taking the absolute value of differences between true and
predicted values and normalizing it with L2 norm of the truth data. We employ
the snapshot-FTLR methodology to recover the pressure force coefficients on the
boundary of the cylinders. Here, CD,p and CL,p denote the drag and lift force
coefficients experienced by the stationary cylinder in a pressure field. Fig. 4.12
depict the effect of reconstruction Ψ on the training forces. These plots are shown
for 75 tU∞ /D till 200 tU∞ /D for the visualisation purpose. The reconstruction
accuracy is nearly 99.5% on the coarse DL grid. Recovering the correct force
from training data boils down to the task of extracting the discrete/pixel force from
accurate predicted fields. The Ψ calculated from the training data recovers the
missing force data in prediction fields. The predicted force coefficients are shown
in Fig. 4.15 for both the cylinders over all the test time steps (3501 till 3800 time
steps).
60
(a)
(b)
(c)

and true fields (CRAN model) along with normalized reconstruction
pressure field.
61
(a)
(b)
(c)

and true fields (CRAN model) along with normalized reconstruction
x-velocity field.
62
(a) (b)
(c) (d)
Figure 4.15: The flow past side-by-side cylinders: Predicted and actual
(CRAN model) pressure force coefficients for all test time steps: (a)
Drag (Cylinder 1), (b) lift (Cylinder 1), (c) drag (Cylinder 2), (d) lift
(Cylinder 2). Note that at a time, one input time step (blue dot) is used
to predict the next sequence of Nt = 25 steps, until a new input is fed.
This helps in reducing the compounding effect of the errors while still
achieving predictions in closed-loop recurrent fashion.
63
4.1.3 Discussion on predictive performance
It is noted that the POD-RNN model with the closed-loop recurrent neural net-
work is inefficient in predicting the flow past side-by-side cylinders. For that pur-
pose, we employ an encoder-decoder type recurrent net instead of a closed-loop
type. Whereas, the convolutional recurrent autoencoder network (CRAN) model
can predict the flow fields in a closed-loop fashion for longer time steps with lim-
ited ground data as a demonstrator. In the case of the POD-RNN model, the spa-
tial and temporal parts of the problem are dealt with independently which might
work for simple problems of flow past a cylinder at low Re. However, such lin-
ear decomposition is less effective in combating highly nonlinear problems such
as side-by-side cylinders. The improvement compared to the POD-RNN model
can be attributed to the low-dimensional features obtained by CNNs of the CRAN
model. The nonlinear kernels in CNNs are very powerful in identifying dominant
local flow features [87]. Also, the complete end-to-end architecture of the network,
which enables us to integrate the encoding, evolution and decoding in a nonlinear
fashion, is the primary reason for this model to work for the problem. This is a very
promising result and motivates us to take forward this concept of convolutional re-
current autoencoder models and its variants (such as variational autoencoders, reg-
istration/interpretable autoencoders [119, 120]) for unsteady fluid flow problems
with strong convection effect and moving boundaries.
Table 4.2 lists out the differences in the attributes of both the models for flow
past side-by-side cylinders for both the fields. It is worth mentioning that CRAN
outperforms POD-RNN in terms of longer time series prediction. In this analysis,
POD-RNN requires ≈ nts /2 = 150 test time steps as input for predicting remaining
150 time steps, while CRAN utilises ≈ nts /Nt = 12 data demonstrators to predict
all nts time steps. Note that nts = 300 denotes the test time steps.
Fig. 4.16 depicts the temporal mean of the reconstruction error Es for both the
models over the test time steps. The temporal mean error Es is quite low in the order
of 10−3 for both the predictions. The majority of the errors are concentrated in the
near wake region where there is the presence of strong nonlinearity. As mentioned
earlier at the start of the discussion, the closed-loop architecture is not effective
for the POD-RNN model hence we need to consider the encoder-decoder structure
64
POD-RNN CRAN
Encoder/decoder space POD θ c ≈ 3 x 105
Evolver features (P,U) k = 25 Nh = 32, 16
Recurrent network type encoder-decoder closed-loop
Training time (CPU) 1h 16 h
Prediction (ni ,p) ni : 25, p : 25 ni : 1, p : 25
θ : trainable parameters
P: pressure field, U: x-velocity field
ni : input time steps, p: predicted time steps
Table 4.2: The flow past side-by-side cylinders: Comparison of the POD-
RNN with convolutional recurrent autoencoder network (CRAN).
where the previous information is required for predicting the future state. In the
case of the CRAN model, a similar performance of encoder-decoder architecture
was obtained with a closed-loop structure. This is one of the advantages of using
the CRAN model over the POD-RNN model.
4.2 Summary
We have presented two data-driven reduced-order models for the prediction of non-
linear fluid flow past bluff bodies. Both methods share a common data-driven
framework. The common principle of both methods is to first obtain a set of
low-dimensional features of the high-dimensional data coming from the fluid flow
problems. The low dimensional features are then evolved in time via recurrent
neural networks. We first use POD to obtain a set of low-dimensional features and
then proceed to a more general method where the low-dimensional features are
obtained by CNNs. Although POD-RNN seems to be simpler in terms of archi-
tecture and implementation, it is not scalable to complex problems. Autoencoders
in the CRAN model are neural network architectures that enable a general way to
construct low-dimensional models via the reconstruction process. Some specific
advantages of the CRAN model over the POD-RNN model are:
• In the case of turbulence, the number of POD modes required to capture the
significant flow physics can be in the order of 104 and more [87]. In the case
65
(a) (b)
(c) (d)
Figure 4.16: The flow past side-by-side cylinders: Comparison between the
temporal mean error Es for POD-RNN and CRAN model. (a),(b) for
pressure and (c),(d) for x-velocity.
of the CRAN model, there is no need of calculating the orthogonal modes.

While POD is a linear transformation of data, the autoencoder process in the
CRAN model is nonlinear and can handle multi-scale and complex dynami-
cal relationships.
• The POD-RNN architecture is not easily scalable to 3D fluid-structure inter-

action problems with moving boundaries, whereas with the CRAN model all
these complexities can be taken care by building deep networks.
66
• The flexibility of neural network layers and their compatibility across multi-
ple deep networks is the main advantage of the CRAN model over the POD-
RNN model.
• POD modes provide the mapping between high-dimensional spatial data to a

low-dimensional embedding in a linear fashion, whereas the same mapping
is achieved via a network of layers in a nonlinear fashion. This linear map-
ping of POD modes can be limited to describing complicated physics. There
is no such constraint of orthogonality in the CRAN model.
For the simple problem of flow past a plain cylinder at low Re, the POD-RNN
model works reasonably well and the CRAN model is a bit overkill for this prob-
lem. However, it performs extremely well and completely bypasses POD-RNN
in terms of accuracy for the complicated problem of flow past side-by-side cylin-
ders. This study shows that convolutional recurrent autoencoders can encompass
a broader range of fluid flow problems and can be pursued further for more com-
plicated problems. Hence, in the following chapters, we extend the CRAN method
to predict nonlinear fluid flow past moving body and three-dimensional flows with
variable Re numbers.
67
Chapter 5
Partitioned Deep Learning for

Fluid-Structure Interaction
In the previous chapter, we presented an assessment of two DL-ROMs, the so-

called POD-RNN and the CRAN, for the flow past side-by-side stationary cylinder
configurations. It was demonstrated that the CRAN framework achieves a longer
time series prediction due to its nonlinear projection space compared with the
POD-RNN, but the former relies on a uniform grid. This uniform grid can lose
the interface description when projecting and interpolating the fluid variables from
the full-order grid. Although the POD-RNN has comparatively lesser flow predic-
tive abilities for complex flows, this model recovers a high-dimensional interface
during the reconstruction process.
In this chapter, we extend the above frameworks for predicting fluid-structure
interaction. For this purpose, we construct a partitioned data-driven approach to
couple the fluid and solid subdomains via deep neural networks. Specifically, the
present chapter unifies the attributes of the POD-RNN and the CRAN frameworks
for reducing the coupled dynamics and inferring the fluid flow interacting with a
freely vibrating structure. Of particular interest is to predict the flow fields via
the CRAN framework, while handling the moving interfaces via the POD-RNN.
Based on an unstructured irregular grid for the full-order simulation, we utilize the
snapshot-field transfer and load recovery (snapshot-FTLR) to select the structured
DL-ROM grid for the fluid domain. We thoroughly investigate the data-driven
68
analysis of VIV motion of a freely oscillating cylinder for a different set of reduced
velocities via a partitioned DL-ROM methodology in this chapter.
5.1 Hybrid partitioned DL-ROM framework

Consistent with the high-dimensional FSI solver, we consider two separate data-
driven solvers for the fluid and solid subdomains with their own independent neural
network in a partitioned manner. Once the high-dimensional data is effectively re-
duced via POD and convolutional autoencoder, the low-dimensional states can be
used to infer the coupled behavior of a fluid-structure system. Together with POD
and convolutional autoencoder for the dimensionality reduction, we use the LSTM-
RNNs for evolving the low-dimensional states in time. The mathematical structure
of the LSTM-RNN represents a nonlinear state-space form. This particular at-
tribute makes them suitable for a nonlinear dynamical system of fluid-structure in-
teraction. When the dimensionality of the data is reduced using POD and evolved
in time using RNN, the hybrid DL-ROM is referred to as the POD-RNN (section
3.4). On a similar note, when the low-dimensional states, obtained using a convo-
lutional autoencoder, are similarly evolved in time, the hybrid framework is called
the CRAN (section 3.5). .
For predicting the coupled dynamics of the fluid flow interacting with a freely
vibrating structure, we unify the combined effects of the POD-RNN and the CRAN
frameworks for the partitioned fluid and solid sub-domains. The illustration of this
hybrid partitioned DL-ROM approach is provided in Fig. 5.1. The first partition or
level of our framework is the POD-RNN for learning the solid displacements, while
the second partition is the CRAN for learning the flow fields in a uniform pixel grid.
On a given training data, the POD-RNN model is employed to track the structural
displacements. This tracking helps in constructing a level-set function. A level-set
function is a distance function based on structural geometry. It is used for setting
the level set boundaries on the computational grid based on the position of the
structure with a binary condition as the input function. Miyanawala & Jaiman [17,
42] presented a CNN model combined with the level-set to predict the unsteady
fluid forces over stationary geometries. Further details about the coupling of a
level-set function with convolutional neural network can be found in [16, 17, 42].
69
𝛀𝐟 : CRAN
𝛀𝐟𝐬
𝐒𝑛 (𝐲ො 𝑛+𝑝 , 𝐒෠ 𝑛+𝑝 )
𝐒 𝑛−1 (𝐲ො 𝑛+𝑝−1 , 𝐒෠ 𝑛+𝑝−1 )
𝐒1
𝐅ത𝑏 INTERFACE LOAD RECOVERY 𝛟
(𝐲ො 𝑛+1 , 𝐒෠ 𝑛+1 )
𝛀𝐬 : POD-RNN
Flow field, rigid body motion & force

predictions
𝐲𝑛
𝐲 𝑛−1
𝐲1 1
Figure 5.1: Illustration of a hybrid partitioned DL-ROM framework for fluid-

structure interaction. The field variables Ωf (t) are learned and predicted
on the uniform grid using the CRAN (blue box), while the interface
information Ωs with moving point cloud is predicted via the POD-RNN
(red box). These boxes exchange the interface information (grey box)
that couples the system via level-set Φ and force signals F̄b . The yellow
box demonstrates the synchronized predictions. As the CRAN learns
the flow variables on the uniform grid (blue box), the level-set Φ is
utilized in calculating the pixelated force signals F̄b on the interface.
An interface load recovery is constructed to select the pixel grid for the CRAN
framework via snapshot-FTLR while extracting the interface information using the
level-set and forces.
The solid and fluid equations are solved sequentially in a partitioned manner
and constrained at the interface with traction and velocity continuity in a parti-
tioned full-order solver for fluid-structure interaction. Analogous to the full-order
partitioned solver, we apply the DL-ROMs sequentially for the solid and fluid vari-
ables and couple them via an interface recovery process. The choice of the so-
70
called physics-based DL drivers can be different and dependent on the nature of
the problem at hand. However, the interface load recovery is general data coupling
for learning and modeling of fluid-structure interaction with pixel-based flow field
information. We next elaborate the methodology for the VIV motion of a freely
oscillating cylinder.
5.2 Freely oscillating cylinder in external flow

This section demonstrates our hybrid partitioned DL-ROM methodology for a
benchmark fluid-structure interaction problem. While the proposed technique can
be generalized to any fluid-structure system involving the interaction dynamics
of flexible structures with an unsteady wake-vortex system, we consider an un-
steady elastically-mounted cylinder problem in an external flow. The wake flow
behind the bluff body generates switching vortex loads on the solid, which causes
its vibration. This structural vibration, in turn, affects the near-wake flow dynam-
ics resulting in a two-way synchronized oscillating system. For this phenomenon
of vortex-induced vibration, the flow physics is highly sensitive and directly re-
lated to the interaction dynamics between the downstream vortex patterns and the
cylinder’s motion. Of particular interest is to learn and predict such synchronized
wake-body interaction using data-driven computing. We consider the parameter
sets corresponding to a limit cycle response as well as non-stationary vibration
response.
The freely vibrating structural system is installed in a 2D computational do-
main. The center of the cylinder is at sufficient distances from the boundaries to
capture the downstream wake dynamics (see Fig. 6.3 (a)). A no-slip and traction
continuity condition is enforced on the cylinder surface (Eqs. (3.4)-(3.5)), while a
uniform velocity profile U∞ is maintained at the inlet Γin . The traction-free bound-
ary is implemented on the outlet Γout and slip condition is specified at the top Γtop
and bottom Γbottom . We discretize the fluid domain via unstructured finite element
mesh. The final mesh, which is obtained after following the standard rules of mesh
convergence, is presented in Fig. 6.3 (b). It contains a total of 25, 916 quadrilat-
eral elements with 26, 114 nodes. The full-order simulation is carried out via finite
element incompressible Navier-Stokes solver in the ALE framework. The output
71
Γ𝑡𝑜𝑝
Γ𝑖𝑛 Γ𝑜𝑢𝑡 30𝐷
15𝐷
Γ𝑏𝑜𝑡𝑡𝑜𝑚
4
10𝐷 20𝐷
(a)
(b)
Figure 5.2: Freely vibrating circular cylinder in a uniform flow: (a)

Schematic of an elastically-mounted cylinder undergoing VIV, (b) rep-
resentative full-order domain and the near cylinder unstructured mesh.
72
Table 5.1: An elastically-mounted circular cylinder undergoing vortex-
induced vibration: Comparison of the full-order forces and the displace-
ments in the present study and the benchmark data. CD and CL represent
the drag and lift force coefficients, respectively. Ax and Ay represent the
x and y displacements of the cylinder, respectively.
∗ N ∗∗

Mesh Ncyl f luid mean(CD ) (CL )rms (Ax )rms Ay max
Present 124 25,916 2.0292 0.1124 0.0055 0.5452

Ref. [36] 168 25,988 2.0645 0.0901 0.0088 0.5548
∗ N
cyl are the number of cylinder surface elements
∗∗ N
f luid are the number of Eulerian fluid elements
details are tabulated in Table 6.1 for validation. By constructing the Cauchy stress
tensor on the fluid-solid surface, the fluid force along the interface is computed.
The drag and lift force coefficients are calculated as
1
Z
CD = 1 f 2
(σ f .n).nx dΓ,
2 ρ U ∞ D Γfs
(5.1)
1
Z
CL = 1 f 2 (σ f .n).ny dΓ,
2 ρ U ∞ D Γfs
where CD and CL denote the drag and lift force coefficients, respectively, and Γfs is
the fluid-solid interface. nx and ny are the Cartesian components of the unit normal
vector n.
5.2.1 Limit cycle vortex-induced vibration

We start by applying our proposed methodology for the VIV at frequency lock-
in. A rigid cylinder is mounted in the transverse and streamwise directions with
linear and homogeneous springs and no damping. This results in alike natural
frequencies fnx = fny in both Cartesian directions. Computation is carried at a fixed
cylinder mass ratio m∗ = 4m U∞
f 2 = 10, reduced velocity Ur = f D = 1
πρ D
U∞
√
n k
= 5,
2π mD
ρ fU∞ D
and Reynolds number Re = µf
= 200. m is the solid mass, fn is the natural
frequency of the cylinder and U∞ is the uniform inlet x-velocity. The remaining
other constants imply their usual meaning. At these non-dimensional parameters,
73
(a) (b)
Figure 5.3: Freely vibrating circular cylinder in a uniform flow (m∗ =

10,Ur = 5, Re = 200): (a) Cumulative and (b) percentage of the modal
energies for the ALE x-displacements.
a high and sensitive limit cycle oscillations are obtained [36].

The full-order simulation is carried out for tU∞ /D = 250 with a time step of
0.025 tU∞ /D. From these full-order data, ntr = 5000 snapshots (100-225 tU∞ /D)
are used for training and nts = 1000 (225-250 tU∞ /D) are kept for testing. Fol-
lowing the data generation, we apply our multi-level framework and the snapshot-
FTLR method for predicting the flow fields, the ALE displacements, and the pres-
sure force coefficients.
Structural displacements via POD-RNN driver

Time instances of the ALE x-displacements or y-displacements Y = y1 y2 . . . yN ∈

Rm×N are decomposed using the POD projection. Here, m = 26, 114 and N = 6000.
′
The fluctuation matrix Y ∈ Rm×N and the offline database of the mean field ȳ ∈ Rm
and the POD modes ν ∈ Rm×N are obtained. Eigenvalues ΛN×N of the covariance
′ ′
matrix Y T Y ∈ RN×N are extracted as their magnitude measures the energy of the
respective POD modes. Cumulative and percentage of the modal energies are plot-
ted in Figs. 5.3 and 5.4 with respect to the number of modes. It can be observed
that the system energy of > 99.99% is concentrated in the first 1 − 2 POD modes.
74
(a) (b)

10,Ur = 5, Re = 200): (a) Cumulative and (b) percentage of the modal
energies for the ALE y-displacements.
This observation implies that the ALE field can potentially be reduced with just
k = 2 POD modes instead of m = 26, 114 fluids nodes. One reason for this obser-
vation is that the majority of the variance about ȳ is concentrated at the moving in-
terface and propagates linearly to the far-field boundaries. Interestingly, with these
two modes, the full-order mesh can be reconstructed within a maximum absolute
error < 1.0 × 10−15 . Fig. 5.5 shows the dynamical evolution of the dominant POD
′
modes ν ∈ Rm×k using Aν = ν T Y ∈ Rk×N . The time coefficients are nearly peri-
odic with a limit cycle oscillation as inherent in the high-dimensional information.
The N temporal coefficients are divided into the training and the testing part.
We train a closed-loop LSTM in a time-delayed fashion for learning these time co-
efficients. This training implies that the predicted output is used as an input for the
next time step. The input to the closed-loop recurrent neural network is a vector of
dimension k so that all k modes are learned and predicted in one time instant. The
nearly periodic trend in the modes allows an easier hyperparameter construction of
the LSTM-RNN, albeit different phases and amplitudes of the modes exist. Further
details about the network can be found in Table 5.2. The training took around 5
75
(a)
(b)

10,Ur = 5, Re = 200): Time history of the k = 2 modal coefficients
shown from 100 till 150 tU∞ /D for the (a) ALE x-displacements and
the (b) ALE y-displacements.
76
Table 5.2: Freely vibrating circular cylinder in a uniform flow (m∗ = 10,Ur =
5, Re = 200): Network and hyperparameters details of the closed-loop
RNN.
ALE k Cell Nh∗ Optimizer α∗ Epochs Learning Decay

rate step
decay
x or y 2 LSTM 256 Adam∗ 0.001 1600 0.2 125
epochs
Nh : LSTM hidden cell dimension
α : Initial learning rate
Adam∗ : Adaptive moment optimization [113]
minutes on a single graphics processing unit, and the prediction of the time coef-
ficients Âν is depicted in Fig. 5.6 for testing. We keep the multi-step prediction
cycle to p = 100 while testing the output. This implies that one test time step is
used to predict the next 100 time coefficients until a ground input is fed. It is worth
mentioning that an encoder-decoder LSTM architecture would have been preferred
to extract the temporal coefficients in the case of chaotic dynamics. In the present
case of periodic oscillations, a simple closed-loop LSTM-RNN is sufficient for the
point cloud tracking.
i
The predicted modal coefficients at any time step i, Âν ∈ Rk , can simply be
reconstructed back to the point cloud ŷi ∈ Rm using the mean field ȳ ∈ Rm and the
k spatial POD modes ν ∈ Rm×k . Fig. 5.7 compares the predicted and true values
of the VIV motion in the x and y-directions of the cylinder. The results indicate
that the POD-RNN can accurately predict the position of the moving FSI interface
for a limit cycle oscillation. These results are inferred in a closed-loop fashion by
predicting p = 100 steps from one demonstrator at a time. The incorporation of
the ground data demonstrators in the predictions can circumvent the compounding
effect of errors and thereby boost the long term predictive abilities of the neural
networks [121]. This description completes the moving point cloud prediction for
the chosen parameter set.
77
(a)
(b)

10,Ur = 5, Re = 200): Prediction of the modal coefficients. (a) ALE
x-displacements and (b) ALE y-displacements.
78
(a) (b)

10,Ur = 5, Re = 200): Predicted and actual (a) x-position and (b) y-
position of the interface normalised by the diameter of cylinder D. The
root mean squared error between the true and predicted is 2.17 × 10−4
and 1.08 × 10−2 for Ax /D and Ay /D, respectively.
Snapshot-FTLR
As mentioned earlier, the POD-RNN driver tracks the moving interface and ex-
tracts the Φ level-set feature. However, the best snapshot DL-ROM grid (Nx × Ny )
must be selected for the field predictions in the CRAN framework. The snapshot-
FTLR estimates the point cloud field data on a uniform grid where a higher-order
load recovery is possible at the practical level. We project the point cloud flow
field data, for instance the pressure fields, s = {s1 s2 ... sN } ∈ Rm×N as spatially
uniform snapshots S = {S1 S2 ... SN } ∈ RNx ×Ny ×N . This uniformity is achieved via
the SciPy’s griddata function [114] by mapping the m dimensional unstructured
data on a 2D uniform grid. Here, Nx = Ny = 32, 64, 128, 256, 512, 1024. Note that
the DL-ROM grid for this case is of the size 8D × 8D with ≈ 5D length for the
downstream cylinder wake. To assess the sensitivity of grid distribution, we com-
pare the accuracy of surface fit provided by the grid data using nearest, linear and
cubic interpolations. This is achieved by sampling the field’s maximum and the
minimum values with pixel numbers on the fitted surface process. As seen from
79
(a) (b)
(c) (d)

10,Ur = 5, Re = 200): Convergence of the pressure field with the
number of pixels for different interpolation schemes at the time step
tU∞ /D = 125. (a)-(b) Maximum pressure values and the respective er-
rors, (c)-(d) minimum pressure values and the respective errors.
Fig. 6.4, for any pressure time step tU∞ /D = 125, the nearest levels-off to the true
pmax and pmin during grid refinement. This reason is that this method assigns the
value of the nearest neighbor in the distributed triangulation-based information.
Remark 2. The linear and cubic interpolation techniques involve triangulations
80
with C0 and C1 continuity. The linear method linearly converges to the true values
on grid refinement. The cubic interpolation approach, however, converges well
on a coarser grid Nx = Ny = 64, 128, but deviates on subsequent grid refinement
Nx = Ny = 256, 512, 1024 with small relative errors.
Fig. 5.9 compares the interpolation methods for the pressure field with respect
to the full-order for tU∞ /D = 125 on the 512 × 512 DL pixelated grid. It can
be observed that the nearest contains oscillations compared to the full-order de-
scription due to a discontinuous assignment of the field at the selected probes. A
closer look in Fig. 5.9 (d) depicts some local fitting deviations compared to the
full-order for the cubic due to a higher-order interpolation. Conversely, the linear
and the nearest look much closer to the full-order results. Hence, we rely on the
linear technique for a coarse-grain field transfer in the remainder of our analysis.
Fig. 5.10 demonstrates the grid dependence of the normalized pressure pixelated
forces Fb /0.5ρ f U∞2 D vs the full-order forces FΓfs /0.5ρ f U∞2 D. We observe that
the interface coarsening leads to the pixelated force propagation to contain some
noises, especially in the direction where the solid displacements are large. These
noisy signals become dominant on reducing the time step and increasing the cell
size. The reasons are attributed to the loss of the interface information due to lower
fidelity and a linear way of the pixelated force calculation. Interestingly, these
noises are somewhat invariant to a sharp fluid-solid interface propagation on a uni-
form grid. These force signals are devoid of any high-frequency noises for the
static boundaries as discussed in in the previous chapter.
To correct these forces, while still maintaining a lower interface grid resolution,
we first apply the Gaussian filtering ζ on the pixelated force propagation to damp
the high-frequency noises. Fig. 5.11 (a)-(b) depict the smooth trend ζ Fb in the
normalized forces on the various DL-ROM grids for the drag and lift coefficients
via a Gaussian filter length of 20. It is worth noting that the pixelation process
results in the mean and derivative variation in the bulk quantities from the full-
order counterpart. These errors reduce with super-resolution. However, we are
interested in refining the uniform grid to an extent where the bulk quantities can be
linearly reconstructed to the full-order within the mean and derivative correction
using Algorithm 1.
81
Full-order grid
(a) (b)
(c) (d)

10,Ur = 5, Re = 200): Qualitative behavior of the different interpola-
tion methods for the pressure field in the ALE reference at the time step
tU∞ /D = 125: (b) Nearest neighbor, (c) linear and (d) piece-wise cubic
type values on the snapshot 512 × 512 uniform grid with respect to (a)
full-order CFD grid.
82
(a) (b)

10,Ur = 5, Re = 200): Total pixelated force propagation (from 100-
125 tU∞ /D) on the snapshot DL-ROM grids vs the full-order for (a)
the pressure drag coefficient CD,p , and (b) the pressure lift coefficient
CL,p on the training field.
The pixelated force and the reconstruction process Ψ on the different resolu-
tions of the DL-ROM grid is summarized in Table 5.3. Columns 3 and 4 tabulate
the mean and the root mean square of the total pixelated drag and lift coefficients,
respectively, for various grid sizes. Column 5 in Table 5.3 similarly denote the
mean of time-dependent derivative correction E c observed from the Algorithm 1.
Columns 6 and 7 depict the reconstruction error ε in the corrected forces ΨFb and
the full-order FΓfs as calculated from step 6 of Algorithm 2. The reconstruction
accuracy is nearly 99.8% and 96.5% for the drag and the lift, respectively, onward
grid 512 × 512. This process facilitates an optimal uniform grid to carry the neural
prediction on while still recovering the bulk quantities within reasonable accuracy.
Remark 3. It is worth mentioning that the propagation of the pixelated force de-
pends on the problem at hand; however, the present reconstruction offers a general
data recovery methodology. Fig. 5.11 (c)-(d) depict the force correction by observ-
ing the Ψ mapping and correcting on the training forces. We select the 512×512 as
the snapshot DL-ROM grid for the flow field predictions. It accounts for a reason-
83
Table 5.3: Freely vibrating circular cylinder in a uniform flow (m∗ = 10,Ur =
5, Re = 200): Pixelated forces and the reconstruction accuracy on the
various snapshot DL-ROM grids with respect to the full-order grid.
Grid Cell-size mean(ζ CD,p )(ζ CL,p )rms mean(E c ) ε(CD,p ) ε(CL,p )
128 0.0625 1.8295 0.1491 0.8534 7.6 ×10−3 3.2 ×10−1

256 0.0313 1.7470 0.1212 0.9124 3.3 ×10−3 1.5 ×10−1
512 0.0157 1.7130 0.1180 0.9307 2.9 ×10−3 4.7 ×10−2
1024 0.0078 1.6979 0.1168 0.9545 2.8 ×10−3 2.8 ×10−2
FOM 2.2 ×10−5 1.6834 0.1174 – – –
able force recovery while avoiding the necessity of super-resolution at the interface
and bypassing the unstructured mesh complexity of the moving point cloud.
Near-wake dynamics using the CRAN driver

With the chosen grid Nx = Ny = 512, the end-to-end nonlinear learning based on
the CRAN is used for the flow field predictions. Algorithm 3 can be employed
to recover the bulk quantities from the field data. As described in section 5.2.1, a
point cloud flow dataset s = s1 s2 . . . sN ∈ Rm×N from the full-order solver is

transferred to the uniform grid using the snapshot-FTLR method. The generated
field information S = S1 S2 . . . SN ∈ RNx ×Ny ×N are decomposed into ntr = 5000

training data (100-225 tU∞ /D) and nts = 1000 testing data (225-250 tU∞ /D). The
training time steps ntr are further sparsed for every 0.05 tU∞ /D, thereby reducing
the total number of trainable steps to ntr /2. This is essentially carried out to speed-
up the CRAN I/O data processing, while still maintaining the same time horizon of
training. As detailed in section 3.5 from Eqs. (3.24)-(3.26), the standard principles
of data normalization and batch-wise arrangement is adopted to generate the scaled
featured input S = Ss′1 Ss′2 . . . Ss′Ns ∈ [0, 1]Nx ×Ny ×Nt ×Ns , where Ns = 100 and

Nt = 25. While training, a mini-batch of hsize ns = 1 is maintained.

i For every
′j
training iteration, a training sample Ss = Ss, j Ss, j . . . Ss, j is randomly shuffled
′1 ′2 Nt
from the scaled featured input to update and refresh the CRAN neural parameters.
The encoding and decoding space of the CRAN is summarized in Table 5.4 in
a way that the spatial downsampling and prolongation are achieved gradually. We
84
(a) (b)
(c) (d)

10,Ur = 5, Re = 200): Interface load behavior vs time (from 100-125
tU∞ /D) on the snapshot DL-ROM grids. (a)-(b) denote the smoothen
force propagation and (c)-(d) depicts recovered interface force infor-
mation for drag and lift coefficients.
85
Table 5.4: Freely vibrating circular cylinder in a uniform flowr (m∗ =
10,Ur = 5, Re = 200): Deep CRAN layer details: Convolutional ker-
nel sizes, numbers and stride. Layers 6,7,8 represent the fully connected
feed-forward encoding, while 9,10,11 depict the similar decoding. The
low-dimensional state Ac is evolved between the layers 8 and 9 with the
LSTM-RNN.
Conv2D encoder DeConv2D decoder

Layer kernel- kernels stride Layer kernel- kernels stride
size size
1 10 × 10 2 4 12 5×5 16 2
2 10 × 10 4 4 13 5×5 8 2
3 5×5 8 2 14 5×5 4 2
4 5×5 16 2 15 10 × 10 2 4
5 5×5 32 2 16 10 × 10 1 4
keep the low-dimensional encoder state Ac and the LSTM-RNN evolver cells of the
sizes: NA = Nh = 32, 64, 128. These modes are obtained from a nonlinear space of
the autoencoder network and are found to be the most sensitive hyperparameter in
this architecture. Since these projections are self-supervised, we experiment with
the different sizes of the modes based on the convergence during training. We
first instantiate the training of all the three CRAN models for Ntrain = 500, 000
iterations starting from the pressure fields. We select the Nh = 128 based on a
faster decay in the overall loss (Eq. (3.27)) compared with the Nh = 32, 64 evolver
states. Nh = 128 pressure model is further optimized until the total iterations reach
to Ntrain = 106 with the overall loss ≈ 4.93 × 10−5 . Once the pressure training is
completed, we transfer the learning to the x-velocity field and optimize the CRAN
network subsequently for Ntrain = 750, 000 iterations.
Remark 4. Each CRAN model is trained on a single graphics processing unit

Quadro GP100/PCIe/SSE2 with Intel Xeon(R) Gold 6136 central processing unit
@ 3.00GHz × 24 processors for nearly 2.7 days. A long training time is considered
so that the CRAN-based framework self-supervises with the maximum optimization.
The training process is stopped when the loss reaches ≈ 1.0 × 10−5 . The test time
steps depict the predictive performance of this network in terms of accuracy check
and remarkable speed-ups compared with the FOM counterpart.
86
Herein, we are interested to demonstrate Nh = 128 trained CRAN model to
infer the flow field with a multi-step predictive cycle of p = Nt = 25. Figs. 5.13 and
5.14 show the comparison of the predicted and the true values of the pressure and x-
velocity fields, respectively, at time steps 9300 (232.5 tU∞ /D), 9600 (240 tU∞ /D)
and 9960 (249 tU∞ /D) on the test data. The normalized reconstruction error E i ,
for any time step say i, is constructed by taking the absolute value of the difference
between the true Si and predicted field Ŝi and normalizing the difference with the
L2 norm of the truth using the equation
|Si − Ŝi |
Ei = . (5.2)
∥Si ∥2
It is evident from these predictions that significant portions of the reconstruction

errors are concentrated in the nonlinear flow separation region of the vibrating
cylinder. These errors are in the order of 10−3 for the pressure and 10−4 for the
x-velocity. The field predictions are accurately predicted in the expected cylinder
motion, albeit with a minor deviation in the reconstruction. We observe that for
the CRAN framework, with one input step, the coupled FSI prediction is accurate
for the next 25 time steps in the future, after which the predictions can diverge.
Thus, the feedback demonstrators combat the compounding errors and enforce the
CRAN predictive trajectory devoid of divergence over the test data.
The synchronous point cloud structural and coarse-grain flow field predictions
are employed to calculate the integrated pressure loads CD,p and CL,p via Algo-
rithm 3. The results are shown in Fig. 5.12 (in the red line). The interface load
algorithm extracts the full-order pressure loads with consistent phases from the
predicted fields on the uniform grid, largely accounting for an accurate flow sepa-
ration and the near-wake predictions. Interestingly, we observe some variations in
the amplitude of the force signals from a direct coarse-grain integration over the
moving interface even though other quantities are predicted well. The reason for
this behavior is the existence of high-frequency noises in the pixelated forces from
the uniform grid, even in the training data. Since the internal training process of the
CRAN does not damp these force noises directly, the predicted fields can also infer
these noises in time. We note that these spurious noises are the largest when the
interface displacement is high and as a result, large residuals are seen in the force
87
(a) (b)

10,Ur = 5, Re = 200): Interface force prediction on the test time steps
from the predictive framework (from 225-250 tU∞ : (a) Pressure drag
coefficient CD,p , and (b) pressure lift coefficient CL,p . The red lines de-
pict the coarse-grain forces obtained from Algorithm 3 and blue lines
depict the corrected forces using a denoising LSTM-RNN.
plots. The development of these residuals in the CNN-based predictions can also
be responsible for a finite amount of predictive ability of the CRAN architecture
and the requirement of ground data to combat this problem.
Remark 5. One potential way to correct these residuals is to dampen the noises
using a denoising recurrent neural network. These networks have been shown to
perform well to denoise signals in a variety of applications such as electrocardio-
graphic signal (ECG) [122] and automatic speech recognition (ASR) [123]. For
demonstration, we damp the noises in the coarse-grain force signals using a de-
noising LSTM-RNN. We achieve this by learning the mapping of the coarse-grained
force signals to the full-order force using a standard LSTM type recurrent network
on finite predicted signals. The filtered signals in Fig. 5.12 (shown in the blue line)
demonstrate good precision for the drag and reasonable in the lift compared to the
full-order. We refer to the works in [124] and [125] for more details.
88
(a)
(b)
(c)

10,Ur = 5, Re = 200): Comparison of the predicted and the true fields
along with the normalized reconstruction error E i at tU∞ /D = (a)
232.5, (b) 240 (c) 249 for the pressure field.
89
(a)
(b)
(c)

10,Ur = 5, Re = 200): Comparison of the predicted and the true fields
along with the normalized reconstruction error E i at tU∞ /D = (a)
232.5, (b) 240 (c) 249 for the x-velocity field.
90
5.2.2 Non-stationary vortex-induced vibration
In section 5.2.1, we discussed the application of our hybrid partitioned DL method-
ology for a VIV response at a mass ratio of m∗ = 10, reduced velocity of Ur = 5
and Reynolds number of Re = 200. Although a high and sensitive VIV response
is obtained at these parameters, the full-order VIV motion is a simple single fre-
quency limit cycle oscillation. Hence, to further strengthen the application of our
methodology, we now apply the partitioned DL-ROM framework for a Ur = 7
while keeping the same Reynolds number and the mass ratio. For the considered
training and testing dataset, the full-order VIV motion consists of small magni-
tudes of non-stationary amplitude responses with no limit cycle oscillation at these
new parameters.
The full-order time series dataset is similarly generated until tU∞ /D = 250. A
total of 2000 snapshots are generated at every 0.125 tU∞ /D for the pressure field
and the ALE displacements. We keep the same time duration of the training and
testing range but with a reduced number of time steps: ntr = 1000 snapshots (100-
225 tU∞ /D) for training and nts = 200 (225-250 tU∞ /D) for testing. At these set
of training and testing range, we note that there is no saturation amplitude in the
dataset, which makes it a good test case to replicate the complexities involved in
the VIV motion that are not necessarily single frequency limit cycle response.
As discussed in section 5.2.1, time instances of the ALE x-displacements or
y-displacements Y = y1 y2 . . . yN ∈ Rm×N for this full-order dataset are decom-

posed using the POD projection in order to reduce the point cloud information.
Here, m = 26, 114 and N = 1200. Cumulative modal energies for the ALE x-
displacements and y-displacements are plotted in Fig. 5.15 (a) and (c), respectively,
with respect to the number of modes. For the chosen parameters, it is interesting to
observe that the POD spectrum for the ALE displacements are still fast decaying
(similar to the limit cycle VIV oscillation in Figs. 5.3 and 5.4) and the majority of
the system energy of > 99.9% can be described by the first 2 POD modes. This
helps in reducing the ALE field with k = 2 POD modes instead of m = 26, 114
fluid nodes for this test case. Notably, with these two modes, the full-order mesh
displacement can be reconstructed within a maximum absolute error < 1.0×10−14 .
91
(a) (b)
(c) (d)

10,Ur = 7, Re = 200): Cumulative modal energies and the time history
of k = 2 modal coefficients shown from 100 till 150 tU∞ /D. (a)-(b) for
the ALE x-displacements, and (c)-(d) for the ALE y-displacements.
Fig. 5.15 (b) and (d) depict the dynamical evolution of the 2 POD modes for the
ALE x-displacements and y-displacements, respectively, obtained from the POD
analysis. As inherent in the high-dimensional VIV motion, we note that these time
coefficients propagate as non-stationary variable amplitude responses with small
frequency differences in every oscillation. The point cloud tracking, hence, reduces
to effective learning and prediction of these dynamical coefficients accurately in
time.
Using a closed-loop LSTM-RNN with the same hyperparameters as in Ta-
92
(a) (b)
(c) (d)

10,Ur = 7, Re = 200): Prediction of the modal coefficients and the
interface motion on the test data. (a) and (c) denote the predicted
and actual time coefficients of the ALE x-displacements and ALE y-
displacements, respectively. (b) and (d) denote the corresponding x
and y position of the interface, normalised by diameter of the cylinder
D. With p = 25, the root mean squared error (RMSE) between the true
and predicted is 5.58 × 10−4 and 2.22 × 10−3 for Ax /D and Ay /D,
respectively. With p = 200, the RMSE between the true and predicted
is 9.11 × 10−4 and 4.44 × 10−3 for Ax /D and Ay /D, respectively.
93
ble 5.2, we learn the POD time coefficients over the 1000 training time steps by
optimising the evolver loss (Eq. 3.20) as detailed in section 3.4. With the training of
5 minutes on a graphics processing unit, the prediction of the modal coefficients is
depicted in Fig. 5.16 (a) and (c) for the ALE x-displacements and y-displacements,
respectively, over the test data. We keep the multi-step prediction cycle of length
p = 25 (one input infers 25 test steps) and p = 200 (one input infers all 200 test
steps). We note that the predictions of these non-stationary time coefficients are in
good agreement with the ground truth with slight improvements in p = 25 predic-
tions as compared to p = 200. The predicted modal coefficients are reconstructed
back to the point cloud using the offline database of the mean field and the POD
modes to track the interface. Fig. 5.16 (b) and (d) depict the comparison of the
predicted and true values of the VIV motion in the x and y directions of the cylin-
der, respectively. The results indicate that the POD-RNN on the ALE motion can
predict the position of the moving FSI interface with excellent accuracy for this
parameter set.
The application of the snapshot-FTLR as a load recovery for this parameter
set is showcased in Fig. 5.17 over the training forces. For the sake of uniformity,
we have selected the same resolution of the snapshot DL-ROM grid 512 × 512 ob-
tained via linear interpolation. Via Gaussian filtering of length 5 and the functional
reconstruction mapping Ψ (using Algorithm 1), the reconstructed force signals ob-
tained show a good agreement for the drag and lift coefficients with the full-order
values. As a result, we transfer the field point cloud to uniform and structured DL
snapshots for the CRAN driver. For demonstration purposes, we only consider the
pressure fields for testing the predictive abilities of the CRAN driver.
The generated pressure field information S = S1 S2 . . . SN ∈ RNx ×Ny ×N us-

ing the snapshot-FTLR are decomposed into ntr = 1000 training data (100-225
tU∞ /D) and nts = 200 testing data (225-250 tU∞ /D). As detailed in section 3.5,
the training flow dataset is normalized and batch-wise arranged to generate the
scaled featured input with number of batches Ns = 40 and the RNN time sequence
Nt = 25. To train the CRAN architecture on this flow dataset, we load the optimized
Nh = 128 CRAN parameters for the pressure model for the limit cycle case in sec-
tion 5.2.1 and optimize further. This transfer of learning helps in optimizing the
CRAN framework for the new parameter set in Ntrain = 50, 000 iterations, which
94
(a) (b)

10,Ur = 7, Re = 200): Recovered interface load behavior vs time
shown from 100-125 tU∞ /D using the snapshot-FTLR for (a) the drag,
and (b) the lift coefficients on the 512 × 512 snapshot DL-ROM grid
with respect to full-order.
took a training time of nearly 7 hours on single graphics processing unit with the
final loss of ≈ 1.0 × 10−5 . The test time steps depict the predictive performance of
this network in terms of accuracy check.
Fig. 5.18 (a) depict the comparison of the predicted and the true values of
the pressure field at the last test time step 250 tU∞ /D obtained with a multi-step
predictive cycle of p = Nt = 25. From these predictions, we note that that the field
predictions are in excellent agreement with the true field, with a reconstruction
error E i in the order of 10−3 . Similar to the limit cycle VIV oscillation, we note
the coupled FSI prediction from the CRAN is accurate for the first 25-30 time steps
in the future, after which the predictions can diverge. The feedback demonstrators
combat the compounding errors and enforce the CRAN predictive trajectory devoid
of divergence over the test data. Figs. 5.18 (b) depict the evolution of the integrated
pressure forces CD,p and CL,p on the interface over the test data using Algorithm 3.
From a direct integration of the predicted fields, we obtain a reasonable drag and
lift prediction. The variation in the predicted amplitude from the force signals are
minor for this case due to a smaller interface displacement in this coupled dataset.
95
(a)
(b)

10,Ur = 7, Re = 200): (a) Comparison of the predicted and the true
fields along with the normalized reconstruction error E i at tU∞ /D =
250 for the pressure field, and (b) interface drag and lift prediction on
the test time steps from the predictive framework with multi-step pre-
dictive cycle p = 25. The CRAN predictions are accurate for the first
25-30 time steps in the future, after which the predictions can diverge.
96
Table 5.5: Freely vibrating circular cylinder in a uniform flow: Summary of
the training and testing times of the DL-ROM components along with
force computation as compared to the full-order computational time. The
testing of the POD-RNN, CRAN and Algorithm 3 denote the run time
needed to predict p = 25 instances of displacements, fields and forces,
respectively. The combined predictions denote the total run time for all
these predictions.
Parameter Training∗ Training∗

(Re = 200, (POD- (CRAN)
m∗ = 10) RNN)
(hours) (hours)
Ur = 5 1.5 × 10−1 6.0 × 101
Ur = 7 1.5 × 10−1 7.5
Parameter Testing∗ Testing∗ Algo. 3∗∗ Combined FOM∗∗ Speed-up

(Re = 200, (POD- (CRAN) (Forces) predictions predictions factor
m∗ = 10) RNN)
(seconds) (seconds) (seconds) (seconds) (seconds)
Ur = 5 2.8 × 10−1 6.4 × 10−1 8.9 × 10−1 1.8 4.3 × 102 2.4 × 102
Ur = 7 3.1 × 10−1 6.5 × 10−1 6.9 × 10−1 1.7 4.3 × 102 2.5 × 102
∗ Single GPU processor
∗∗ Single CPU processor
As a result, the noises in the forces are of a smaller magnitude.

Finally, we provide an estimate of computational cost for the hybrid partitioned
DL-ROM together with full-order simulations. In Table 5.5, we report the training
times, the testing times and the total time needed to predict the solid displacements,
the flow predictions and the forces (using Algorithm 3). We recall that the multi-
level DL-ROM solution can offer remarkable speed-ups in computing as compared
with the FOM, by nearly 250 times. While a canonical laminar flow of vibrating
cylinder is considered, the proposed hybrid DL-ROM does not make any assump-
tions with regard to geometry and boundary conditions.
97
5.3 Summary
In this chapter, we have presented a hybrid partitioned deep learning framework for
the reduced-order modeling of moving interfaces and fluid-structure interaction.
The proposed novel DL-ROM relies on the proper orthogonal decomposition com-
bined with recurrent neural networks and convolutional recurrent autoencoder net-
work. We decouple structural and fluid flow representations for easier and modular
learning of two physical fields independently. Using coarse to fine-grained learn-
ing of fluid-structure interaction, a low-dimensional inference of the flow fields and
the full-order mesh description with load recovery has been discussed.
By analyzing a prototype FSI model for limit cycle and non-stationary struc-
tural motions, we have first shown that the POD-RNN component infers the point
cloud dynamics by projecting the ALE displacements on the POD modes. A
closed-loop LSTM-RNN effectively propagates the evolution of the POD time co-
efficients and tracks the full-order grid motion accurately. We then analyze an itera-
tive low interface resolution DL-ROM grid search for the CNNs that preserves the
full-order pressure stresses on the moving fluid-structure interface via snapshot-
FTLR. We have shown that this snapshot-FTLR method selects the grid for the
CRAN architecture but can exhibit spurious oscillations and data loss in the loads
on a moving FSI system. These errors are shown to depend on the uniform grid
resolution, and a detailed study has been performed to recover and filter the losses.
A 16-layered trained CRAN network is shown to predict the flow dynamics
involving a coupled FSI with accurate estimates of flow prediction. The CRAN
network extrapolates the field accurately for 25-30 time steps from one input data,
after which the predictions diverge owing to some noises in the CRAN reconstruc-
tion. This leads to instability in the force calculation on the moving interface.
Once trained, the proposed DL-ROM technique can be implemented using only
a fraction of computational resources to that of a full-order model for online pre-
dictions. The proposed framework paves way to the development of partitioned
digital twin of engineering systems, especially those involving moving boundaries
and fluid-structure interactions. In the next chapter, we investigate the data-driven
predictions for a realistic three-dimensional unsteady flow dynamics with variable
Re.
98
Chapter 6
Three-dimensional Deep
Learning for Parametric
Unsteady Flows
In this chapter, we turn our attention towards realistic three-dimensional unsteady

flow predictions via hybrid DL-ROM. Of particular interest is to study parametric
flow and force predictions with variable Reynolds numbers. Throughout the chap-
ter, we utilize the convolutional autoencoder process for general reduced-order
modeling. We consider flow past a static sphere to examine the accuracy of our
data-driven predictions. This chapter introduces the voxel-based data generation
process via snapshot-FTLR from the full-order 3D flow fields. Next, the DL-
ROM methodology employing the convolutional recurrent autoencoder network
and transfer learning is presented for efficient 3D training. The implementation of
the proposed methodology for flow past a sphere with a single Re is first described.
Subsequently, the case of variable Re-based flow information is examined to learn
a complicated symmetry-breaking flow regime (280 ≤ Re ≤ 460) for the flow past
a sphere. Finally, we provide a detailed discussion on the variation of predicted
forces to Re and data-driven computational speed-ups of complex 3D wake flow.
99
CFD unstructured field 3D DL uniform mesh
𝐬 x, y, z, t ∈ ℝ𝑚 𝐒 X, Y, Y, t ∈ ℝ𝑁𝑥 ×𝑁𝑦 ×𝑁𝑧
𝑁𝑧
Interface cells
Field transfer
𝐬(𝑡) 𝐒(𝑡)
Solid cells
Load recovery 𝑁𝑦
𝚿𝐅ത𝑏 (𝑡) 𝐅ത𝑏 (𝑡)
Fluid cells
y
z x 𝑁𝑥
• The point cloud to voxel data reduces unstructured mesh complexity in the DL grid
Figure 6.1: Schematic of a 3D mesh-to-mesh field transfer and load recovery
• Using the level-set
processfunction, the voxelthe
to determine forces
3Dare computed
CRAN from interface
snapshot cells to the details
grid. Refer
• The load recovery is achieved by the mean
of all variables in section 6.1.and derivative correction using Ψ mapping
• This allows preserving the higher order interface information on DL grid
• 𝑚 ≈ 700𝑘 and 𝑁𝑥 , 𝑁𝑦 , 𝑁𝑧 = 64
6.1 Field transfer and coarse-graining
To retain the spatial connectivity information in the full-order flow dataset, we3
project the unstructured field dataset on a uniform voxel grid to coarse-grain the
field information as shown in Fig. 6.1. This simple process brings uniformity in the
field information together with convenience in training the 3D CRAN driver. The
projection and interpolation of information are achieved via the snapshot-FTLR
process described in section 3.6. Its extension to 3D is highlighted briefly in the
following sub-sections.
6.1.1 Field transfer

We extend the snapshot-FTLR to 3D flow fields throughout this chapter. Let s =
{s1 s2 ... sn } ∈ Rm×n be the full-order flow fields generated from the Navier-Stokes
1 2 n
solver and FΓfs = {FΓfs FΓfs ... FΓfs } ∈ R3×n be the corresponding interface forces
over the fluid-solid body along the three Cartesian directions. Here, m represent the
number of full-order nodes and n denotes the number of snapshots. The point cloud
dataset at every time step is interpolated and projected to a uniform voxel grid via
SciPy’s griddata function [114] as shown in Fig. 6.1. The size of the uniform grid
can be chosen Nx × Ny × Nz .
100
6.1.2 Load recovery
The presence of the interface in the DL voxel grid is ensured by masking the entire
3D spatial region covered by the static solid domain. This identifies the coarse in-
terface voxels (shown as yellow cells in Fig. 6.1) that contain the solid description.
1 2 n
As detailed in section 3.6, we first integrate the total voxel force Fb = {Fb Fb ... Fb }
exerted by these interface cells over the solid using
NF Z
i
Fb = ∑ σki .ndΓ, i = 1, 2, ..., n, (6.1)
k=1 Γk
where NF are the number of interface voxels. For every interface cell k, σki is the
Cauchy stress tensor at a time step t i . We calculate this tensor using the finite
difference approximation and integrate over the faces of the voxel Γk .
Grid coarsening leads to a loss of accurate forces on the physical interface,
even in the training data. This is accounted due to a considerable loss of interface
resolution in the voxel grid as compared with the FOM mesh. We recover the
1 2 n
data loss in the voxel forces Fb = {Fb Fb ... Fb } by mapping to full-order FΓfs =
1 2 n
{FΓfs FΓfs ... FΓfs } while maintaining a lower DL grid resolution. This is achieved
by constructing the functional recovery mapping Ψ that is described by Algorithm
1. We select the coarse grid Nx ×Ny ×Nz that recovers the bulk force quantities with
Ψ mapping by avoiding the need of super-resolution. The entire 3D mesh-to-mesh
field transfer and load recovery process brings uniformity in the field information
to coarse-grain the full-order dataset. It also recovers the interface information
together with providing convenience in training the CRAN driver.
6.2 Deep transfer learning for DL-ROMs

Scaling the CRAN to 3D is algorithmically straightforward as the framework largely
relies on 3D CNNs to extract the flow features. However, this simple algorith-
mic extension can considerably increase the memory requirement, hyperparameter
space, and training costs for the end-to-end learning model. In some cases, the
offline training time for learning a simple flow regime can take a matter of days or
weeks if the CRAN framework starts learning from scratch. For example, in the
previous chapter, we saw that the 2D CRAN takes around 2.7 days to be trained for
101
Source (Single 𝑹𝒆) Target (Variable 𝑹𝒆)
𝑹𝒆 𝑹𝒆𝟏 𝑹𝒆𝟐 𝑹𝒆𝐯

(𝑛 steps) (𝑛v steps) (𝑛v steps) … (𝑛v steps)
Training
Prediction
DS Source domain
DS 3D CRAN Transfer learning 3D CRAN DT TS Source task
DT Target domain
TT Target task
TS 𝑹𝒆: 𝑹𝒆𝟏 𝑹𝒆𝟐 𝑹𝒆𝒗
(𝑝 steps) (𝑝 steps) … TT
(𝑝 steps) (𝑝 steps)
Figure 6.2: Illustration of the transfer learning process for training variable
Re flows. Source refers to learning and extrapolating single Re flows in
time. Target refers to learning and extrapolating multiple Re flows in
time. Note that for training steps nv < n.
FSI. This often results in: (a) enormous computing power for training than the full- 12
order model itself and, (b) large training data requirement for the neural network.
These challenges can complicate the process of training a 3D CRAN-based frame-
work, especially for learning complex flow patterns involving variable Re-based
flows.
To overcome these challenges, transfer learning is beneficial. Transfer learning,
employed in machine learning, refers to the use of a previously trained network to
learn a new task. In transfer learning, a machine uses a previously learned task
to increase generalisation about another. The neural parameters and task for a
pre-trained network are called as the source domain and source task, respectively.
Whereas, the neural parameters and task for a new network are called as the target
domain and target task, respectively. This is further elaborated in Fig. 6.2. For a
source domain DS with a corresponding source task TS and a target domain DT
with a corresponding task TT , transfer learning is the process of improving the
target CRAN predictive function by using the related information from DS and TS ,
where DS ̸= DT or TS ̸= TT . The single source domain defined here can be extended
to multiple target domains or tasks. This study employs transfer learning to train
the 3D CRAN for a variable Re flow regime on a limited data and training time.
102
For this purpose, we load a pre-trained model of a single Re case and optimize for
variable Re flows. The source domain and task are to learn and predict single Re
flows in time using 3D CRAN. On the flip side, the target domain and task become
learning and prediction of multi-Re flows in time using one 3D CRAN.
6.2.1 Training and prediction

We use the same CRAN architecture and, training and prediction algorithms for
variable Re flows as described in section 3.5. The only difference is the prepara-
tion of the dataset matrix and utilization of 3D CNNs instead of 2D. For variable
Re flows, the time series data are snapshots of flow fields for different Re values
acquired from the full-order simulation. Consider that Rem = [Re1 Re2 ... Rev ] be
the range of Reynolds number for generating the full-order fields. The training
dataset matrix S(Rem ) consists of time series data for different Re fields (pressure
or velocity) that are stacked as a matrix
 1 

[S (Re1 ) S2 (Re1 ) · · · Snv (Re1 )],
 
[S1 (Re2 ) S2 (Re2 ) · · · Snv (Re2 )],
 
S(Rem ) = .. . (6.2)


 . 



 1 
[S (Rev ) S2 (Rev ) · · · Snv (Rev )]

Note that S(Rem ) ∈ RNx ×Ny ×Nz ×(vnv ) consists of v Reynolds numbers and each con-
sisting nv snapshots. Nx = Ny = Nz = 64 represents the number of data probes for
the uniform voxel grid. The training dataset matrix is normalised and batch-wise
arranged for ease in training. The matrix re-arranges in the following form
S (Rem ) = Ss′1 Ss′2 . . . Ss′Ns ∈ [0, 1]Nx ×Ny ×Nz ×Nt ×Ns ,

(6.3)
h i
where each training sample Ss′ j = S′1 S ′2 . . . SNt is a time series data at a partic-
s, j s, j s, j
ular Re value. Nt are the evolver steps while Ns are the number of batches. While
training, sequence of different Re field is selected randomly Ss′ j ⊂ S (Rem ) to
optimize the loss function given by Eq. (3.27).
The prediction algorithm utilizes closed-loop evolution of the LSTM-RNN.
For a given initial 3D flow snapshot Snv (Rei ) ∈ [0, 1]Nx ×Ny ×Nz at a Reynolds num-
103
ber say Rei ⊂ Rem , the trained 3D CRAN parameters and Eq. (3.22) is applied
iteratively for p steps with Anc v (Rei ) ∈ Rh as the initial solution. This helps to infer
multiple-Re fields in time for a chosen value of Rei . With transfer learning, a 3D
CRAN model can be built with comparatively less training data and time because
the model is already pre-trained. This can be valuable in tasks where the data can
be limited and unlabeled, for instance variable Re flows.
6.3 Flow predictions past a sphere

In this section, we test our proposed 3D snapshot-FTLR and DL-ROM method-
ologies for data-driven prediction of flow past a sphere. We are interested in in-
tegrating an end-to-end 3D spatial encoding-decoding and temporal evolution for
a realistic CFD problem with usual boundary conditions. Of particular interest is
to forecast flow fields for single and variable Re flow information in the DL space
using an optimized CRAN framework, while preserving the interface description
from the voxel grid.
A schematic of the problem configuration employed for the full-order data
generation ΩCFD , of a stationary mounted sphere, is shown in Fig 6.3 (a). The
sphere system of diameter D is installed in the 3D computational domain of size
50D × 20D × 20D, with center at sufficient distances from the far-field bound-
aries to capture the downstream sphere wake. u, v and w depict the streamwise,
transverse and vertical flow velocities in the x, y and z directions, respectively. A
uniform free-stream velocity profile {u, v, w} = {U∞ , 0, 0} is maintained at the in-
let boundary (Γin ). Here, the free stream velocity is adjusted by defining the Re
of the problem using Re = ρ fU∞ D/µ f , with ρ f and µ f being the fluid density and
viscosity, respectively. Along the top Γtop , bottom Γbottom , and side surfaces, a slip-
wall boundary condition is implemented while a traction-free Neumann boundary
is maintained on the outlet Γout . The streamwise Cx , the transverse Cy and the
vertical force coefficients Cz on the submerged sphere are calculated by integrating
104
the Cauchy stress tensor σ f on the sphere Γfs .
1
Z
Cx = 1 f 2
(σ f .n).nx dΓ,
2 ρ U ∞ D Γfs
1
Z
Cy = 1 f 2 (σ f .n).ny dΓ, (6.4)
2 ρ U ∞ D Γfs
1
Z
Cz = 1 f 2 (σ f .n).nz dΓ,
2 ρ U∞ D Γ
fs
where nx , ny and nz are the x, y and z Cartesian components, respectively, for the
unit normal n of the surface.
In the following sub-sections, we apply the 3D snapshot-FTLR and CRAN
methodologies for synchronously predicting the flow fields and the pressure force
coefficients by selecting a DL domain of interest ΩDL as shown in Fig. 6.3(a).
6.3.1 Unsteady flow at constant Reynolds number

To assess the 3D flow reconstruction and coarse-grain field predictions, we first
examine our 3D data-driven DL-ROM framework on an unsteady fully submerged
sphere problem in external flow at a single Re. The objective, herein, is to learn the
strength and shedding orientation of unsteady planar-symmetric flow at Re = 300,
where the downstream hair-pinned shaped vorticesqshed strongly periodic in the
near sphere wake. The drag Cx and total lift CL = C2y + C2z coefficients demon-
strate periodic behavior pattern from tU∞ /D ≈ 200 onward. The full-order un-
steady flow simulation is first carried out in the unstructured CFD domain to gen-
erate the full-order data. We use a time step of ∆t = 0.05 tU∞ /D for a total of
400 tU∞ /D at Re = 300. The final mesh is obtained using standard principles of
mesh convergence and the full-order output details are tabulated in Table 6.1 to
validate the FEM solver.
A total of 1600 time snapshots of point cloud data are saved at every 0.25 tU∞ /D
for the pressure and x-velocity field. From these full-order data, n = 1000 snap-
shots (from 95-345 tU∞ /D) are kept for training and nts = 100 (from 345-370
tU∞ /D) are reserved for testing. Thus, the total time steps in this analysis are
N = 1100. We further organize the test data in groups of every p = 20 time steps
to assess the compounding error effects in the multi-step predictions from the 3D
105
𝐳
𝐱 𝛤top
𝐲 10𝐷
𝐷
Ω𝐶𝐹𝐷
𝛤in
𝑢 = 𝑈∞ (𝑹𝒆) Ω𝐷𝐿 8𝐷 𝛤out
10𝐷
𝑣=0
8𝐷
𝑤=0 5𝐷
10𝐷
𝛤bottom
10𝐷
10𝐷 40𝐷
(a)
(b)
Figure 6.3: (a) Schematic and associated boundary conditions of flow past a 2
sphere and deep learning domain of interest. (b) Representative CFD

mesh for the entire domain sliced in Z/D = 10 plane.
Table 6.1: The flow past a sphere: The present study’s full-order force values
compared to benchmark data. CD and CL represent the mean drag and
lift force coefficients, respectively. St is the Strouhal number.
Study CD CL St
Present 0.669 0.082 0.137
Johnson and Patel 0.656 0.069 0.137
[126]
106
(a)
4 5
(b)
Figure 6.4: The flow past a sphere: (a) Pressure field convergence with num-
ber of flow voxels for interpolation scheme variants. ε(.) is the respec-
tive relative error. (b) Descriptive behaviour of nearest-neighbour (mid-
dle) and linear interpolation (right) techniques for pressure field in the
DL space 128 × 128 × 128 with respect to CFD space (left) sliced at
Z/D = 10. Plots correspond to tU∞ /D = 200.
CRAN solver. After generating the full-order point cloud dataset, we apply the 3D
CRAN framework to forecast the flow fields past a sphere in a DL-based voxel grid
for Re = 300 while preserving the exact interface description.
We employ the snapshot-FTLR to bring field uniformity in the DL space,
while recovering forces as described in section 6.1. The field uniformity reduces
the model-order fidelity and unstructured mesh complexity by mapping the m-
dimensional unstructured dataset on a 3D reference grid. We compare the field
interpolation methods provided by griddata [114]: nearest and linear methods,
107
with respect to the number of flow voxels. This is performed by sampling the
field’s maximum and minimum values for various 3D DL grid resolution at an in-
stant tU∞ /D = 200. On DL grid refinement, the nearest method levels-off to the
true pmax and pmin for a pressure instant as illustrated in Fig. 6.4(a). Because this
method assigns the value of the nearest neighbour in the unstructured information,
this effect is expected. The linear interpolation approach, however, linearly con-
verges pmax and pmin to the true full-order values on grid refinement. Fig. 6.4(b)
compares the field interpolation methods for pressure field with respect to the full-
order on a 128 × 128 × 128 DL voxel grid. The presence of the sphere boundary
is ensured by masking the exact interface description in the 3D DL grid. It can
be interpreted that, because of a discontinuous assignment of fields at the specified
probes, the nearest method contains oscillations compared to the full-order descrip-
tion. With the linear interpolation and projection, a nearly perfect match is obtained
in terms of the descriptive near wake snapshot (ε(pmax ) ≈ 7%, ε(pmin ) ≈ 2%) de-
void of noises. The qualitative description is further substantiated by the conver-
gence behavior in Fig. 6.4(a). Hence, we rely on the linear interpolation technique
for 3D coarse-grain field assignment.
1 2 N
The total voxel force propagation Fb = {Fb , Fb , ..., Fb } (drag and lift force
components) are obtained from the interface voxels on various 3D DL Cartesian
grids using Eq. (6.1). Fig. 6.5 demonstrates the DL grid dependence of normal-
ized pressure voxel forces Fb /0.5ρ f U∞2 D and their data recovery effects using the
mapping Ψ. It can be interpreted that low-resolution leads to mean and derivative
errors in the voxel forces compared with the full-order CFD forces. The primary
reasons are the considerable loss of boundary fidelity in the DL grid and a linear
force calculation using finite difference. Fig. 6.5 also depicts the force correc-
tion by observing the Ψ mapping and correcting on the training forces. For the
flow field predictions using 3D CRAN, we rely on the DL grid 64 × 64 × 64 be-
cause it accounts for a reasonable force recovery across all components (ε(Cx,p ) =
0.0019, ε(CL,p ) = 0.0026) without requiring the need of super-resolution. With
super-resolution, the voxel force errors are indeed decreased. However, we want
to refine the DL grid to the point where the force coefficients can be transformed
to the full-order with mean and derivative error quantifications. This process facil-
itates an optimal uniform grid to carry the neural prediction. The snapshot-FTLR
108
Figure 6.5: The flow past a sphere: Voxel interface force propagation and
load recovery effects on various snapshot 3D DL grids (shown from
170-245 tU∞ /D). Voxel drag and lift components (Row 1). Corre-
sponding voxel force recovery (Row 2). Red, blue and green dashed
lines represent the grid 32 × 32 × 32, 64 × 64 × 64 and 128 × 128 × 128,
respectively. The black line depicts the full-order force.
procedure is scalable for voxel grid selection so that 3D CNNs can be conveniently
integrated with the point cloud full-order dataset.
With the chosen DL grid Nx = Ny = Nz = 64, the 3D CRAN is employed
for the coarse-grain flow field predictions. The coarse-grain pressure information
S = S1 S2 . . . SN ∈ RNx ×Ny ×Nz ×N is decomposed into n = 1000 learning data (95-

345 tU∞ /D) and nts = 100 analysis data (345-370 tU∞ /D). Following this, standard
principles of data normalization and batch-wise arrangement are adopted to gener-
ate the 5D scaled featured input S = Ss′1 Ss′2 . . . Ss′Ns ∈ [0, 1]Nx ×Ny ×Nz ×Nt ×Ns

109
Table 6.2: Comparison of computational resources used for the 3D CRAN
and 2D CRAN training.
3D CRAN 2D CRAN
(flow past sphere) (flow past cylinder)
DL grid 64 × 64 × 64 64 × 64
Training snapshots n = 1000 n = 1000
RAM (gB) 32 16
Processor number Single GPU node Single CPU node
Processor type NVDIA v100 Intel E5 2690
Trainable parameters θ ≈ 6 × 106 θ ≈ 3 × 105
Mini-batch size ns = 1 − 2 ns = 2 − 5
Training time 64 h 16 h
with Ns = 40 and Nt = 25. The encoding space of the 3D CRAN encodes the
64 × 64 × 64 flow voxel-based input dimension via four layers of 3D CNN oper-
ation with a constant kernel size of 5 × 5 × 5 and stride 2 × 2 × 2. Every CNN
operation reduces the input size by half, with number filters increasing by twice in
every layer. Three feed-forward networks further map the feature vectors until a
finite low-dimensional encoding Ac ∈ RNh is achieved with Nh << Nx × Ny × Nz .
Since 3D CNNs can considerably increase the trainable variables, the cost of hy-
perparameter tuning and training is very high. This can result in an increase in
computing power for training the framework. Table 6.2 depicts a comparison of
the computational resources used for the 3D CRAN training and the 2D CRAN.
We note that scaling the CRAN architecture to three-dimension increases the train-
able parameters by an order magnitude with the increase in random access memory
(RAM) and training time.
To train the 3D CRAN, we experiment with different sizes of the evolver cells
Nh = 64, 128, 256 as primary tuning hyperparameters. In the 3D CRAN, the low-
dimensional evolution needs to be tuned for appropriate time series learning and
iterative optimization. We start by training each 3D CRAN model on a single v100
graphics processing unit (GPU) on pressure fields by instantiating with random
parameter values. These parameters are updated in every training iteration. Every
training iteration consists of a mini-batch of size ns = 2 randomly shuffled from the
110
Figure 6.6: The flow past a sphere: Evolution of the loss function Eh with
training iterations for different evolver cell sizes Nh . P and U denote the
3D CRAN trained with pressure and x-velocity datasets, respectively.
Blue and red dots depict the saved instances for testing the pressure and
x-velocity fields, respectively. The blue cross represents the initializa-
tion of velocity training from saved pressure parameters.
scaled featured flow input S and updating the neural parameters in ≈ 0.3s. This
helps speed-up the training procedure of the deep 3D CRAN architecture and lower
memory usage. The objective function consists of the hybrid loss obtained from
the unsupervised-supervised training as detailed in section 3.5.1. The evolution of
the objective function Eh with the training iterations Ntrain is showcased in Fig. 6.6.
It can be observed that Nh = 256 CRAN model tunes for the pressure dataset
in 6 × 105 training iterations at a loss of 1.74 × 10−6 , which took nearly 64 hours
of GPU training. At these training costs and iterations, 3D CRAN models Nh =
64, 128, however, do not optimise on the pressure dataset. We save the optimised
3D CRAN model (Nh = 256) at Ntrain = 600800 on the disc memory as trained
parameters. This is shown as blue dot in Fig. 6.6. To avoid the expensive hyper-
parameter search for the velocity field training, we instead load the saved pressure
3D CRAN parameters on the velocity dataset (shown as blue cross) and optimise
111
it further. This transfer of learning depicts that the 3D CRAN model fine-tunes
on the coupled velocity dataset and mimics the dynamical model of flow past a
sphere. This initialisation of trained weights to velocity field reduces the training
time to nearly 2 hours. We save the x-velocity 3D CRAN model parameters at
Ntrain = 700000 for velocity testing. This model is depicted by red dot in Fig. 6.6.
Herein, Nh = 256 trained saved instances of the 3D CRAN models are em-
ployed to analyze the field predictions for the pressure and x-velocity on the test
dataset (100 time steps from 345 − 370 tU∞ /D). We keep the multi-step predictive
cycle length of p = 20, 100, implying that one input step infers p sequence of fu-
ture field predictions until a new input is fed. Figs. 6.7 and 6.8 depict a comparison
of the predicted and true values of pressure and x-velocity fields, respectively, at
tU∞ /D = 365 sliced in various orthogonal planes with p = 100. The normalized
reconstruction error E i is calculated by taking the absolute value of the difference
between the real and predicted fields and then normalizing it with the truth’s L2
norm. It can be observed that the majority of these errors are located in the nonlin-
ear flow separation region and near-wake of the sphere. These 3D reconstruction
errors are in the order of 10−3 for pressure and 10−4 for x-velocity predictions.
This demonstrates the high accuracy of the 3D CRAN for reconstruction and time
series extrapolations if properly trained.
The predicted coarse-grain flow fields are directly integrated and corrected us-
ing the snapshot-FTLR to get the pressure loads over the sphere. Evolution of the
drag Cx,p and lift CL,p loads are depicted in Fig. 6.9. The red line in the figure
depicts force calculation from pressure fields with sequence prediction length of
p = 20 steps in a closed-loop recurrence. This helps reduce the compounding ef-
fect of the errors with slight improvements compared to p = 100 steps predicted
from one time instant. In Table 6.3 we provide an estimate of the computational
costs for the 3D CRAN together with the full-order simulations for flow past a
sphere. We recall that the DL-ROM solution can offer remarkable speed-ups in
online predictions compared with the FOM by nearly 1800 times. However, the
offline training time of 3D CRAN is expensive compared with a similar FOM. The
following sub-section extends our DL-ROM methodology for predicting unsteady
flow fields with variable Re while focusing on speeding up both offline and online
computational times.
112
Figure 6.7: The flow past a sphere: Predicted and true pressure field com-
parison along with normalized reconstruction error E i at tU∞ /D = 365
sliced in Z/D = (8, 10) (Row 1), Y/D = (10, 12) (Row 2), X/D =
(10, 12) (Row 3). Left, middle and right contour plots depict the predic-
tion, true and errors, respectively.
113
Figure 6.8: The flow past a sphere: Predicted and true x-velocity field com-
parison along with normalized reconstruction error E i at tU∞ /D = 365
sliced in Z/D = (8, 9.75) (Row 1), Y/D = (10.25, 12) (Row 2), X/D =
(10.5, 12) (Row 3). Left, middle and right contour plots depict the pre-
diction, true and errors, respectively.
114
(a) (b)
Figure 6.9: The flow past a sphere: Predicted and actual (3D CRAN model)
(a) drag and (b) lift force coefficients integrated from the predicted pres-
sure field on the sphere for all 100 test time steps with multi-step pre-
dictive sequence p = 20 and p = 100.
Table 6.3: Summary of the offline and online times for 3D CRAN vs. 3D
FOM simulations.
FOM-HPC 3D CRAN-PC
Processor number 32 CPUs 1 GPU
Offline time∗ ≈ 10 h ≈ 64 h
Online time∗∗ ≈1h ≈ 1.99 s
Offline speed-up 1 0.1563
Online speed-up 1 1800
∗ Elapsed time 1000 training steps.
∗∗ Elapsed time 100 test steps.
6.3.2 Unsteady flow with variable Reynolds number

As seen previously, a trained 3D CRAN model and snapshot-FTLR offer fast and
accurate data-driven field predictions and physical force integration. By fast and
accurate, we mean that one can avoid running the full-order model at a specific
Re and replace it with an optimized 3D CRAN framework. However, the major
bottleneck of this deep learning architecture is the task of hyperparameter tuning
and training, even for learning a periodical vortex shedding phenomenon at con-
115
𝜖 𝐂𝐱,𝑝 = 3.0453e − 04 𝜖 𝐂𝐱,𝑝 = 0.0020 𝜖 𝐂𝐱,𝑝 = 0.0026
𝜖 𝐂𝐋,𝑝 = 0.0019 𝜖 𝐂𝐋,𝑝 = 0.0095 𝜖 𝐂𝐋,𝑝 = 0.0125
4 6 8
𝜖 𝐂𝐱,𝑝 = 0.0031 𝜖 𝐂𝐱,𝑝 = 0.0025 𝜖 𝐂𝐱,𝑝 = 0.0028

𝜖 𝐂𝐋,𝑝 = 0.0160 𝜖 𝐂𝐋,𝑝 = 0.0168 𝜖 𝐂𝐋,𝑝 = 0.0324
10 12 14
𝜖 𝐂𝐱,𝑝 = 0.0029 𝜖 𝐂𝐱,𝑝 = 0.0012 𝜖 𝐂𝐱,𝑝 = 0.0024

𝜖 𝐂𝐋,𝑝 = 0.0275 𝜖 𝐂𝐋,𝑝 = 0.0259 𝜖 𝐂𝐋,𝑝 = 0.0287
15 17 19
Figure 6.10: The variable flow past a sphere: Recovered voxelated load prop-
agation (pressure drag and lift) on DL grid 64 × 64 × 64 vs full-order
CFD grid for variable Re flows. Dashed and solid lines indicate the
recovered DL grid loads and full-order loads, respectively.
stant Re. The expensive offline training for another Re-dependent flow is, hence,
less appealing for data-driven prediction from a practical standpoint. Moreover,
the training can become challenging in flow scenarios that involve multi-Re infor-
mation. Subsequently, in this sub-section, we explore the training and predictive
abilities of the 3D CRAN framework with multiple Re-based flow patterns. Of par-
ticular interest is to optimize a 3D CRAN framework on a variable Re flow dataset
within acceptable training cost and accurate predictive abilities.
We start by generating the full-order unsteady point cloud dataset for a variable
Re-based 3D flow regime. We utilize the CFD domain in Fig. 6.3 to generate
flow snapshots for Rem = [280 300 320...460] with a time step 0.25 tU∞ /D. For
116
every Re ⊂ Rem , we select a reduced time series training dataset nv = 400 (from
250 till 350 tU∞ /D). However, we maintain the same number of testing dataset
nts = 100 (from 350 till 375 tU∞ /D) as compared to a single Re scenario. While
the hairpin shaped vortices are periodically shed for the unsteady planar-symmetric
flow regime 280 ≤ Re ≤ 370, as the Re is increased, the shedding orientation of the
unsteady hairpin vortices becomes asymmetric in 370 ≤ Re ≤ 460. The particular
flow regime makes the problem challenging and is a good test case to replicate
complexities in flow phenomenon where Re can change. In the present case, we are
interested in learning the strength and shedding of 3D flow patterns from unsteady
planar-symmetric to asymmetric flows for 280 ≤ Re ≤ 460.
The point cloud field dataset is processed by interpolating and projecting in the
same uniform DL space ΩDL of size 8D × 8D × 8D using the snapshot-FTLR.
Like the single-Re flow scenario, the coarse-grain interpolation and projection
of the unstructured dataset are achieved using the linear method on a voxel grid
(Nx × Ny × Nz ) = (64 × 64 × 64). The voxel forces from the 3D DL space are cor-
rected by observing functional corrective mapping Ψ for every Re-based flow in-
formation. Fig. 6.10 depicts the voxel force corrections on the training forces, with
respective reconstruction error ε(.) for the drag and lift signals over the sphere. It
can be interpreted that the mean and derivative error corrections over various Re
numbers on the same 3D DL grid account for the generality of the snapshot-FTLR
data recovery process. Irrespective of the flow patterns, the FTLR method recovers
the bulk forces within ε(Cx,p ) = 0.003 for drag and ε(CL,p ) = 0.03 for lift, with-
out requiring a change in the DL grid or grid resolution. Analogous to a single
mesh generation process in CFD applications, the snapshot-FTLR method poten-
tially generates a uniform DL grid for the domain-specific problem. Moreover, the
inherent unstructured mesh complexity can be bypassed by focussing on a uniform
Eulerian grid and 3D CNN operations.
The full-order training dataset matrix S(Rem ) ∈ RNx ×Ny ×Nz ×(vnv ) consists of v =
10 Re numbers and each Re consisting nv = 400 flow snapshots. The scaled flow
trainable input S (Rem ) = Ss′1 Ss′2 . . . Ss′Ns ∈ [0, 1]Nx ×Ny ×Nz ×Nt ×Ns is generated

using the basic principles of normalisation

h andibatch-wise arrangement with Ns =
160, Nt = 25. Note that Ss′ j = S′1 ′2 ′Nt
s, j Ss, j . . . Ss, j is a time series data at a particular
Re value. The complete spatio-temporal training dataset for the present case is in
117
Figure 6.11: The variable flow past a sphere: Evolution of the loss function
Eh with training iterations for different evolver cell sizes Nh . P and
U denote the 3D CRAN models trained with variable Re-based pres-
sure and x-velocity datasets, respectively. The blue cross and red cross
represent the initialization of pressure and x-velocity training from op-
timized single Re 3D CRAN model from Fig. 6.6. Blue and red dots
depict the new saved instances of the 3D CRAN parameters.
the order of 1.04 × 109 . The dataset preparation is detailed in section 6.2. To train
such a big spatio-temporal dataset on a deep 3D CRAN architecture, we initialize
the network training from saved Re = 300 parameters as source domain. This
is done to gain the advantage of fine-tuning the 3D CRAN for variable Re flow
data from single Re flow parameters and bypassing the expensive hyperparameter
search. This is further elaborated in Fig. 6.11 where the evolution of the hybrid
loss function Eh is showcased with the training iterations. Every training iteration
consists of a mini-batch of size ns = 1 randomly shuffled from the scaled flow input
S (Rem ) and updating the neural parameters in ≈ 0.15s.
118
Figure 6.12: The variable flow past a sphere: Predicted and true pressure field
comparison with normalized reconstruction error E i at tU∞ /D = 372.5
sliced in Z/D = 10 plane. Left, middle and right contour plots depict
the prediction, true and errors, respectively.
As shown in Fig. 6.11, the 3D CRAN architecture with Nh = 64, 128, 256 does
not optimise with a random parameter search for the pressure dataset even after
training for ≈ 64 hours on a single GPU. However, the transfer of learning im-
proves traditional learning by transferring knowledge learned in a single Re flow
scenario and improving learning in variable Re flow scenarios. With transfer learn-
ing, the 3D CRAN starts to optimize from Ntrain = 10000 iterations. We save
119
the new 3D CRAN model parameters after Ntrain = 80000 for the pressure and
x-velocity testing with Eh = 2.73 × 10−6 and 3.90 × 10−5 , respectively. The to-
tal offline training time for learning variable Re flow regime took around 3 hours
of single GPU training by leveraging single Re flow domain knowledge. For the
data-driven predictions of the pressure and x-velocity, the trained 3D CRAN mod-
els with Nh = 256 are utilized. We keep the multi-step predictive cycle length
of p = 25. The predicted and true values of the pressure field at the test time
372.5 tU∞ /D (80th step) are compared in Fig. 6.12 where the contour plots for
Z/D = 10 plane are shown. From the reconstruction and inference, it is interesting
to observe that the network differentiates and infers in time a specific Re-based
field that it is instantiated with. The reconstruction error E i is calculated by taking
the absolute value of the difference between the true and predicted fields and then
normalizing it with the truth’s L2 norm for the 3D DL space. The errors are in
the order of 10−2 near the interface and 10−4 elsewhere and are found to increase
slightly in the nonlinear wake as the flow becomes asymmetric from Re ≥ 360.
These predictions imply that the network accurately learns the low-dimensional
patterns for the variable Re-based flow with limited and unlabelled information.
Similarly, the profiles of the streamwise velocity from the predicted fields and
ground truth are compared in Fig. 6.13 at test time step 372.5 tU∞ /D in Z/D = 10
plane. Closed-loop predictions at all Re are in good agreement with the ground
truth velocity in terms of the peak, width, and shape of the streamwise velocity
profiles. Velocity profiles at Y/D = 9.0, 9.5 show no identifiable differences be-
tween the ground truth and 3D CRAN predictions at all Reynolds numbers. This is
because flow at Y/D = 9.0, 9.5 is almost laminar layer flow, the characteristics of
which are relatively easily trained by the network. Minor differences in the veloc-
ity deficit are observed for Y/D = 10.0 in the nonlinear wake region for Re ≥ 380
where the 3D CRAN does not accurately capture small-scale oscillatory motions.
120
𝟏𝟎. 𝟎𝟎
𝟗. 𝟓𝟎
𝟗. 𝟎𝟎
(a) 3
(b)
Figure 6.13: The variable flow past a sphere: (a) Predicted (left) and true
(right) x-velocity field comparison. (b) Comparison of the streamwise
velocity profiles of the 3D CRAN prediction and ground truth at three
locations Y/D = 9.0, 9.5, 10 for all Reynolds numbers. Results are
plotted at test time tU∞ /D = 372.5 sliced in Z/D = 10 plane. Circles
indicate the 3D CRAN predictions, and solid lines represent the ground
truth.
121
(a)
(b)
Figure 6.14: The variable flow past a sphere: 3D CRAN and FOM compar-
ison of (a) mean drag and (b) mean lift variation over the sphere for
different Reynolds numbers.
Based on the predicted pressure flow fields and the snapshot-FTLR force in-
tegration, we discuss the comparison of the mean drag and lift forces from the
3D CRAN prediction and ground truth. Fig. 6.14 shows the performance of the
CRAN-based force predictions when fed with different 3D flow snapshots for
Reynolds number Re ⊂ Rem . To compare the accuracy of the force predictions,
122
Table 6.4: Summary of the offline and online times for 3D CRAN vs. 3D
FOM simulations for variable Re-based flow.
FOM-HPC 3D CRAN-PC
Processor number 32 CPUs 1 GPU
Offline time∗ ≈ 50 h ≈ 2.5 h
Online time∗∗ ≈ 10 h ≈ 20 s
Offline speed-up 1 20
Online speed-up 1 1800
∗ Elapsed time 4000 training steps for 10 Re.
∗∗ Elapsed time 1000 test steps for 10 Re.
we report the R2 error between the true Ci and predicted Ĉi mean force coefficients
calculated using
2
2 ∑ Ĉi − C̄
R = 1− 2 . (6.5)
∑ Ci − C̄
Here, Ci can be the mean drag or lift for a particular Reynolds number. The R2
errors for the mean drag and lift fit for different Reynolds numbers are 98.58% and
76.43%, respectively, which demonstrates the high efficiency of the CRAN-based
prediction process. We find that the predictions perform the best when the field
values correspond to Re ≤ 380, which characterizes a 3D symmetric shedding of
the unsteady vortex patterns. Furthermore, when Re ≥ 440, all the predictions are
accurate within a 5% error margin of the FOM results. The performance of the
3D CRAN-based deep learning becomes slightly deficit over the 3D transitional
flow regime consisting of Re = 400, 420. Interestingly, the maximum prediction
errors correspond to this complicated flow from symmetric to asymmetric unsteady
patterns. The data of a similar problem may enhance the accuracy of predictions in
this transitional flow regime. The most significant result is that the 3D CRAN has
accurately captured the maximum and minimum mean drag and lift coefficients for
the chosen flow regime 280 ≤ Re ≤ 460 in a dearth of training data and on limited
training time. Accurate force predictions correspond to a proper field inference
from the 3D CRAN framework.
Finally, in Table 6.4, we provide an estimate of the computational costs for
123
the 3D CRAN together with the full-order simulations for flow past a sphere with
variable Reynolds number. We recall that the DL-ROM solution offers remarkable
speed-ups in online predictions and offline training in this case. Compared to a 32
CPU parallel FOM solver, a single 3D CRAN framework learns the variable Re
flow regime 20 times faster via the transfer learning process. At the same time, the
online predictions achieved are 1800 times faster than the parallel FOM solver.
6.4 Summary
In this chapter, a deep learning-based reduced order model (DL-ROM) has been
employed for predicting the fluid forces and unsteady vortex shedding patterns
past a sphere. We have presented the DL-ROM methodology based on three-
dimensional convolutional recurrent autoencoder network (3D CRAN) to extract
low-dimensional flow features from the full-order snapshots in an unsupervised
manner. These low-dimensional features are evolved in time using a long short-
term memory-based recurrent neural network (LSTM-RNN) and reconstructed back
to the full-order as flow voxels. Using the snapshot-FTLR, we have shown that the
flow voxels can be effectively introduced as static and uniform query probes in the
point cloud domain to reduce the unstructured mesh complexity. The 3D CRAN-
based DL-ROM methodology has been successfully applied to an external flow
past a static sphere at single Re = 300 with excellent 3D flow reconstruction and
inference. We have provide a detailed assessment of the computing requirements
in terms of the memory usage, training costs and testing times associated with the
3D CRAN framework. The case of variable Re-based flow information has also
been examined to learn a complicated symmetry-breaking flow regime (280 ≤ Re
≤ 460) for the flow past a sphere. Along with transfer learning, the 3D CRAN
framework learns the complicated flow regime nearly 20 times faster than the par-
allel full-order model and predicts this flow regime in time with a good accuracy.
Based on the predicted flow fields, the network demonstrates an R2 accuracy of
98.58% for the drag and 76.43% for the lift over the sphere in this flow regime.
The proposed framework paves the way for the development of a digital twin for
3D unsteady flow field and instantaneous force predictions with variable Re effects.
124
Chapter 7
Conclusions and
Recommendations
In this chapter, the conclusions of the present dissertation and recommendations

for future studies in the present areas of research are summarised.
7.1 Conclusions
In the present dissertation, we have developed and investigated a series of data-
driven methods via deep learning-based reduced-order models. The dynamical
prediction of unsteady fluid flows, fluid-structure interaction and three-dimensional
parametric-dependent flow problems are thoroughly studied.
We have developed an overall framework for reduced-order modeling com-
bined with deep learning to solve the nonlinear dynamical prediction of unsteady
flows. A general end-to-end framework for predictive modeling is developed by
combining high-fidelity data, reduced-order models, and neural networks. The
principle idea behind the hybrid DL-ROM frameworks relies on extracting the
low-dimensional features from the full-order snapshots and learning the dynamic
evolution of the low-dimensional features via nonlinear methods such as recur-
rent neural networks. Two hybrid DL-ROMs are introduced, which vary in the
method of obtaining the low-dimensional features, i.e., POD or convolutional au-
toencoders. In the first model, the high-dimensional data is decomposed into the
125
low-dimensional modes and the temporal coefficients as the reduced-order dynam-
ics using the Galerkin projection of POD. The time coefficients are propagated
using variants of LSTM-RNNs: closed-loop or encoder-decoder type. This hy-
brid technique is termed the POD-RNN. The POD-RNN model is a semi non-
linear model in obtaining the full-order flow state from the POD modes and the
corresponding modal coefficients. In the second model, an autoencoder-based non-
projection reduced-order model is introduced where convolutional neural networks
obtain the low-dimensional features. Closed-loop recurrent neural networks are
employed to learn the temporal evolution of the low-dimensional features. This ap-
proach via convolutional recurrent autoencoder serves as an end-to-end nonlinear
model which can overcome the Kolmogorov n-width issue of the POD-Galerkin
methods. We have also developed a novel data processing utility that combines
unstructured CFD data with the structured grid for convolutional neural networks.
This strategy of the snapshot-field transfer and load recovery preserves the inter-
face description and loads on a general CNN grid.
Both POD-RNN and CRAN have been applied to predict unsteady flows and
instantaneous forces to benchmark data-driven flow predictions past static bluff
bodies. For this purpose, we have systematically assessed the unsteady flow pre-
dictions for a configuration of side-by-side cylinders with the wake interference.
It is noted that the hybrid POD-RNN model with a closed-loop recurrent neural
network did not perform well when applied to predict flow past side-by-side cylin-
ders. Although 25 POD modes captured 95% of the system energy, the highly
chaotic and nonlinear nature of the POD time coefficients hindered the capability
of a closed-loop recurrent neural network to extrapolate in time. For that pur-
pose, an encoder-decoder (seq2seq) recurrent network performed reasonably well
for predicting the POD time coefficients by encoding a series of input data. In
contrast, the convolutional recurrent autoencoder network has been demonstrated
to predict the chaotic flow fields in a closed-loop fashion for longer time steps with
limited ground data as a demonstrator. From our study, we concluded that CRAN
outperforms POD-RNN by 25 times in terms of longer time series prediction for
unsteady flows past static bodies. In our data-driven analysis, the POD-RNN re-
quired 150 test time steps as input for predicting the remaining 150 time steps,
while the CRAN utilized 12 inputs to predict all 300 time steps. In the case of
126
the CRAN model, a similar performance of the encoder-decoder architecture was
obtained with a closed-loop structure. This is one of the advantages of using the
CRAN model over the POD-RNN model for general unsteady flow prediction.
For the time series prediction of fluid-structure interaction, we have developed
a hybrid partitioned deep learning framework for the reduced-order modeling of
moving interfaces and fluid-structure interaction. The proposed novel DL-ROM
relies on the proper orthogonal decomposition combined with recurrent neural net-
work and convolutional recurrent autoencoder network. We decouple structural
and fluid flow representations for easier and modular learning of two physical fields
independently. While the POD-RNN provides an accurate extraction of the fluid-
structure interface, the CRAN enables the extraction of the fluid flow fields. We
have successfully demonstrated the inference capability of the proposed DL-ROM
framework by predicting the time series of the unsteady pressure field of an FSI set-
up of an elastically-mounted circular cylinder. Using coarse to fine-grained learn-
ing of fluid-structure interaction, a low-dimensional inference of the flow fields and
the full-order mesh description with load recovery has been discussed. By analyz-
ing a prototype FSI model for limit cycle and non-stationary structural motions, we
have first shown that the POD-RNN component infers the point cloud dynamics by
projecting the ALE displacements on the POD modes. A closed-loop LSTM-RNN
effectively propagates the evolution of the POD time coefficients and tracks the
full-order grid motion accurately. We then analyze an iterative low interface res-
olution DL-ROM grid search for the CNNs that preserves the full-order pressure
stresses on the moving fluid-structure interface via snapshot-FTLR.
We have shown that this snapshot-FTLR method selects the grid for the CRAN
architecture but can exhibit spurious oscillations and data loss in the loads on a
moving FSI system. These errors are shown to depend on the uniform grid reso-
lution, and a detailed study has been performed to recover and filter the losses. A
16-layered trained CRAN network is shown to predict the flow dynamics involving
a coupled FSI with accurate estimates of flow prediction. The CRAN network ex-
trapolates the field accurately for 25-30 time steps from one input data, after which
the predictions diverge owing to some noises in the CRAN reconstruction. This
leads to instability in the force calculation on the moving interface. The applica-
tion of the CRAN on the flow fields with a moving interface has been successfully
127
demonstrated by preserving the forces on a 512 × 512 CNN grid. The network per-
forms reasonably well in predicting unsteady flows with a moving interface, but
requires a large initial training time and ground data demonstration to combat the
errors in the predictions. We note that the errors are dominant for the parameter set
where the interface displacement is the largest and causes stability issues in the pre-
dictions of fields and forces. The FSI predictions from the model are satisfactory
but there is scope for further improvement. Once trained, the proposed DL-ROM
technique can be implemented using only a fraction of computational resources to
that of a full-order model for online predictions. The FSI time-series predictions
are shown to speed up by 250 times in online prediction compared to FOM.
Finally, we have presented a deep learning framework for the reduced-order
modeling of realistic three-dimensional unsteady flow, emphasizing variable Re-
based flow dynamics. The proposed 3D DL-ROM framework relies on the con-
volutional autoencoder with recurrent neural networks for data-driven predictions.
While the 3D CNNs provide accurate extraction of the low-dimensional features
from full-order flow snapshots, the closed-loop LSTM-RNN enables the propaga-
tion of the features in time. We have successfully demonstrated the inference ca-
pability of the proposed 3D CRAN framework by predicting the time series of the
unsteady flow fields of three-dimensional flow past a sphere. Using coarse-grained
learning of Re-dependent unsteady flows, a low-dimensional inference of the flow
fields with interface load recovery has been discussed. We have first analyzed an
iterative low interface resolution voxel grid search for the 3D CNNs that preserves
the full-order pressure stresses on the fluid-structure interface via snapshot-FTLR.
We have shown that this snapshot-FTLR method selects a coarse-grain grid for the
CRAN architecture by bringing field uniformity and recovering 3D interface infor-
mation. This reduces the point cloud complexity and the number of nodes in CNN
space by three times compared to CFD space. An end-to-end 3D CRAN is shown
to predict the flow dynamics with accurate estimates of flow prediction at single
and variable Reynolds numbers.
By analyzing an external flow problem past a sphere, we have shown that the
3D CRAN infers and reconstructs the flow fields remarkably for Re = 300. The
3D CRAN extrapolates the field from one input data but requires an expensive of-
fline training cost and hyperparameter search. The hyperparameter search has been
128
found to be sensitive to the size of the low-dimensional state of the autoencoder and
a detailed study has been performed to tune the network. For the first time, we have
demonstrated the learning and inference capabilities of the 3D CRAN on a compli-
cated symmetry-breaking flow regime (280 ≤ Re ≤ 460) for the flow past a sphere.
By leveraging the trained parameters for a single Re, we have shown that 3D CRAN
can be trained for a variable Re flow regime on a limited data and training time.
Using the process of transfer learning, we achieve the offline training speed-up by
nearly 20 times compared to the parallel full-order solver. We find that the predic-
tions perform the best when the field values correspond to Re ≤ 380, which char-
acterizes a 3D symmetric shedding of the unsteady vortex patterns. Although the
network performs reasonably well for asymmetric flows, the maximum prediction
errors correspond to the transitional flow regime from symmetric to asymmetric
vortex shedding. The 3D CRAN offers nearly three order times faster predictions
of 3D flow fields with variable Reynolds numbers using a fraction of training time.
In summary, novel methods for data-driven modeling are developed and ex-
plored in this thesis via deep learning and reduced-order modeling. These methods
are applied to various canonical problems related to bistable flows, fluid-structure
interaction and 3D parametric unsteady flow. The results show promising capabili-
ties for such methods for real-time fluid flow predictions and multi-query analysis.
7.2 Recommendations
We have provided several recommendations for the future extension of the present
dissertation. They are summarized as follows:
• Because of the modular, scalable, and adaptable qualities, the CRAN frame-
work introduced in this dissertation has a lot of potential as a data-driven pre-
diction tool for 3D unsteady fluid flow and fluid-structure interaction prob-
lems. It should be applied to convection-dominated or turbulence problems
characterized by large Kolmogorov n-width.
• In the current thesis, the data for the CRAN is acquired from CFD simula-
tions using the snapshot-FTLR. However, images from experimental study
can also be used in the presented framework. The CRAN framework can be
129
easily applied to flow images coming from experiments to predict real-time
flows and instantaneous forces.
• The input data demonstrators in the CRAN architecture can be reduced by

using attention-based CNNs, encoder-decoder RNN architecture or modifi-
cation of loss functions in the neural network.
• The partitioned DL-ROM methodology provides easier and modular learn-

ing of two physical fields independently. It should be further refined and
applied to multiphysics problems such as cavitating or two-phase flows in
offshore and marine applications.
• The POD-RNN is applied for rigid body equations involving a simplified

VIV motion in this thesis. One could extend the framework to flexible struc-
tures with geometric and/or material nonlinearity via the CRAN driver.
• Instead of POD, hyperreduction techniques such as energy-conserving sam-

pling and weighting [107] can be used for evolving nonlinear structural dy-
namics. Moreover, neural architectures such as graph neural networks [64]
and transformers [127] can be considered within the CRAN and partitioned
DL-ROM methodology.
• In this work, the DL-ROM methodology models vortex-induced vibration in

a 2D non-body conformal mesh. It is worth considering a 3D extension of
the DL-ROM methodology for FSI and an adaptive refinement/coarsening
of the non-interface cells to reduce the number of unknowns.
• Instead of CNNs, graph neural networks [64, 109] can be utilized to incor-
porate non-Euclidean geometry, efficient interface resolution and topology
changes for FSI and multiphase flows.
• The 3D CRAN framework along with transfer learning has a great poten-
tial for real-time parametric-dependent unsteady flow prediction. It should
be further pursued for real-time online adaption and generalizing of unseen
Reynolds numbers.
130
Bibliography
[1] R. Gupta and R. Jaiman. A hybrid partitioned deep learning methodology

for moving interface and fluid–structure interaction. Computers & Fluids,
233:105239, 2022.
[2] R. Gupta and R. Jaiman. Three-dimensional deep learning-based reduced

order model for unsteady flow dynamics with variable reynolds number.
arXiv preprint arXiv:2112.09302, 2021.
[3] S. R. Bukka, R. Gupta, A. R. Magee, and R. K. Jaiman. Assessment of

unsteady flow predictions using hybrid deep learning based reduced-order
models. Physics of Fluids, 33(1):013601, 2021.
[4] E. Glaessgen and D. Stargel. The digital twin paradigm for future nasa and
us air force vehicles. In 53rd AIAA/ASME/ASCE/AHS/ASC structures,
structural dynamics and materials conference 20th AIAA/ASME/AHS
adaptive structures conference 14th AIAA, page 1818, 2012. → pages
xiii, 2
[5] D. Hummels. Transportation costs and international trade in the second era
of globalization. Journal of Economic perspectives, 21(3):131–154, 2007.
→ page 1
[6] J. J. Corbett and J. J. Winebrake. International trade and global shipping.

Handbook on Trade and the Environment, pages 33–48, 2008. → page 1
[7] Z. Wan, M. Zhu, S. Chen, and D. Sperling. Pollution: three steps to a green
shipping industry. Nature News, 530(7590):275, 2016. → page 1
[8] C. Erbe, S. A. Marley, R. P. Schoeman, J. N. Smith, L. E. Trigg, and C. B.

Embling. The effects of ship noise on marine mammals—a review.
Frontiers in Marine Science, 6:606, 2019. → page 1
131
[9] C. M. Duarte, L. Chapuis, S. P. Collin, D. P. Costa, R. P. Devassy, V. M.
Eguiluz, C. Erbe, T. Gordon, B. S. Halpern, H. R. Harding, et al. The
soundscape of the anthropocene ocean. Science, 371(6529), 2021. → page
1
[10] M. A. McDonald, J. A. Hildebrand, and S. M. Wiggins. Increases in deep
ocean ambient noise in the northeast pacific west of san nicolas island,
california. The Journal of the Acoustical Society of America, 120(2):
711–718, 2006.
[11] P. H. Dahl, J. H. Miller, D. H. Cato, and R. K. Andrew. Underwater
ambient noise. Acoustics Today, 3(1):23–33, 2007. → page 1
[12] M. F. McKenna, S. M. Wiggins, and J. A. Hildebrand. Relationship
between container ship underwater noise levels and ship design,
operational and oceanographic conditions. Scientific reports, 3(1):1–10,
2013. → page 1
[13] C. Erbe, A. MacGillivray, and R. Williams. Mapping cumulative noise
from shipping to inform marine spatial planning. The Journal of the
Acoustical Society of America, 132(5):EL423–EL428, 2012. → page 1
[14] J. A. Hildebrand. Anthropogenic and natural sources of ambient noise in
the ocean. Marine Ecology Progress Series, 395:5–20, 2009. → page 1
[15] S. R. Bukka, A. R. Magee, and R. K. Jaiman. Deep convolutional recurrent
autoencoders for flow field prediction. arXiv preprint arXiv:2003.12147,
2020. → pages 2, 17, 24
[16] T. P. Miyanawala and R. K. Jaiman. A hybrid data-driven deep learning
technique for fluid-structure interaction. In International Conference on
Offshore Mechanics and Arctic Engineering, volume 58776, page
V002T08A004. American Society of Mechanical Engineers, 2019. →
pages 3, 16, 17, 23, 69
[17] T. P. Miyanawala and R. K. Jaiman. An efficient deep learning technique
for the navier-stokes equations: Application to unsteady wake flow
dynamics. arXiv preprint arXiv:1710.09099, 2017. → pages
2, 13, 15, 27, 69
[18] V. Joshi and R. K. Jaiman. A hybrid variational Allen-Cahn/ALE scheme
for the coupled analysis of two-phase fluid-structure interaction.
International Journal for Numerical Methods in Engineering, 117(4):
405–429, 2019. → pages 2, 12, 13
132
[19] F. Tao, H. Zhang, A. Liu, and A. Nee. Digital twin in industry:
State-of-the-art. IEEE Transactions on Industrial Informatics, 15(4):
2405–2415, 2018. → page 2
[20] S. R. Bukka, A. R. Magee, R. K. Jaiman, J. Liu, W. Xu, A. Choudhary, and

A. A. Hussain. Reduced order model for unsteady fluid flows via recurrent
neural networks. In International Conference on Offshore Mechanics and
Arctic Engineering, volume 58776, page V002T08A007. American Society
of Mechanical Engineers, 2019. → pages 3, 29
[21] R. K. Jaiman and V. Joshi. Computational Mechanics of Fluid-Structure

Interaction. Springer, 2022. → page 3
[22] I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. MIT press,

2016. → pages 4, 14
[23] Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. nature, 521(7553):

436, 2015. → pages 4, 14
[24] D. H. Wolpert. The lack of a priori distinctions between learning

algorithms. Neural computation, 8(7):1341–1390, 1996. → page 4
[25] W. K. Blake. Mechanics of flow-induced sound and vibration, Volume 2:

Complex flow-structure interactions. Academic press, 2017. → page 6
[26] J. Carlton. Marine propellers and propulsion. Butterworth-Heinemann,

2018. → page 6
[27] R. Arndt, P. Pennings, J. Bosschers, and T. Van Terwisga. The singing

vortex. Interface focus, 5(5):20150025, 2015. → page 6
[28] D. Wittekind and M. Schuster. Propeller cavitation noise and background

noise in the sea. Ocean Engineering, 120:116–121, 2016. → page 6
[29] S. R. Kashyap and R. K. Jaiman. A robust and accurate finite element

framework for cavitating flows with fluid-structure interaction. Computers
& Mathematics with Applications, 103:19–39, 2021. → pages 6, 12
[30] D. H. Lim and K. S. Kim. Development of deep learning-based detection

technology for vortex-induced vibration of a ship’s propeller. Journal of
Sound and Vibration, page 116629, 2021. → page 7
[31] Q. Zhang and R. K. Jaiman. Numerical analysis on the wake dynamics of a

ducted propeller. Ocean Engineering, 171:202–224, 2019. → page 7
133
[32] P. S. Gurugubelli and R. K. Jaiman. Self-induced flapping dynamics of a
flexible inverted foil in a uniform flow. Journal of Fluid Mechanics, 781:
657–694, 2015. → page 12
[33] V. Joshi, R. K. Jaiman, and C. Ollivier-Gooch. A variational flexible
multibody formulation for partitioned fluid–structure interaction:
Application to bat-inspired drones and unmanned air-vehicles. Computers
& Mathematics with Applications, 80(12):2707–2737, 2020. → pages
12, 13
[34] R. K. Jaiman, M. Z. Guan, and T. P. Miyanawala. Partitioned iterative and
dynamic subgrid-scale methods for freely vibrating square-section
structures at subcritical reynolds number. Computers & Fluids, 133:68–89,
2016. → pages 12, 13, 20
[35] V. Joshi, P. S. Gurugubelli, Y. Z. Law, R. K. Jaiman, and P. F. B.
Adaikalaraj. A 3d coupled fluid-flexible multibody solver for offshore
vessel-riser system. In International Conference on Offshore Mechanics
and Arctic Engineering, volume 51210, page V002T08A009. American
Society of Mechanical Engineers, 2018. → page 12
[36] Z. Li, W. Yao, K. Yang, R. K. Jaiman, and B. C. Khoo. On the
vortex-induced oscillations of a freely vibrating cylinder in the vicinity of a
stationary plane wall. Journal of Fluids and Structures, 65:495–526, 2016.
→ pages 12, 73, 74
[37] J. Donea. Arbitrary Lagrangian–Eulerian finite element methods.
Computational Methods for Transient Analysis, pages 474–516, 1983. →
page 13
[38] J. A. Sethian. Level set methods and fast marching methods: evolving
interfaces in computational geometry, fluid mechanics, computer vision,
and materials science, volume 3. Cambridge university press, 1999. →
page 13
[39] C. S. Peskin. The immersed boundary method. Acta numerica, 11:
479–517, 2002. → page 13
[40] Z. Yu. A dlm/fd method for fluid/flexible-body interactions. Journal of
computational physics, 207(1):1–27, 2005. → page 13
[41] D. Mokbel, H. Abels, and S. Alanda. A phase-field model for
fluid–structure interaction. Journal of computational physics, 372:
823–840, 2018. → page 13
134
[42] T. P. Miyanawala and R. K. Jaiman. A novel deep learning method for the
predictions of current forces on bluff bodies. In International Conference
on Offshore Mechanics and Arctic Engineering, volume 51210, page
V002T08A003. American Society of Mechanical Engineers, 2018. →
pages 13, 15, 69
[43] T. P. Miyanawala and R. K. Jaiman. A low-dimensional learning model via

convolutional neural networks for unsteady wake-body interaction. arXiv
preprint arXiv:1807.09591, 2018. → page 14
[44] J. Schmidhuber. Deep learning in neural networks: An overview.

https://arxiv.org/abs/1404.7828, 2014. → page 14
[45] J. Schmidt-Hieber. The kolmogorov–arnold representation theorem

revisited. Neural Networks, 137:119–126, 2021. → page 14
[46] G. Cybenko. Approximation by superpositions of a sigmoidal function.

Mathematics of Control, Signals, and Systems, 2(4):303–314, 1989. →
page 14
[47] T. Chen and H. Chen. Universal approximation to nonlinear operators by

neural networks with arbitrary activation functions and its application to
dynamical systems. IEEE Transactions on Neural Networks, 6(4):911–917,
1995.
[48] K. Hornik, M. Stinchcombe, and H. White. Universal approximation of an

unknown mapping and its derivatives using multilayer feedforward
networks. Neural networks, 3(5):551–560, 1990. → page 14
[49] M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković. Geometric deep

learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint
arXiv:2104.13478, 2021. → page 14
[50] A. Goyal and Y. Bengio. Inductive biases for deep learning of higher-level
cognition. arxiv.org/pdf/2011.15091.pdf, 2021. → page 14
[51] S. R. Bukka. Data-driven computing for the stability analysis and

prediction of fluid-structure interaction. PhD thesis, 2019. → pages 14, 23
[52] Y. Ham, J. Kim, and J. Luo. Deep learning for multi-year enso forecasts.
Nature, 573(7775):568–572, 2019. → page 15
[53] M. E. Brown, D. J. Lary, A. Vrieling, D. Stathakis, and H. Mussa. Neural

networks as a tool for constructing continuous ndvi time series from avhrr
135
and modis. International Journal of Remote Sensing, 29(24):7141–7158,
2008.
[54] O. San and R. Maulik. Machine learning closures for model order
reduction of thermal fluids. Applied Mathematical Modelling, 60:681–710,
2018. → page 15
[55] H. Chen, Y. Zhang, M. K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou, and

G. Wang. Low-dose ct with a residual encoder-decoder convolutional
neural network. IEEE transactions on medical imaging, 36(12):
2524–2535, 2017. → page 15
[56] S. Lunz, O. Öktem, and C. Schönlieb. Adversarial regularizers in inverse

problems. In Advances in Neural Information Processing Systems, pages
8507–8516, 2018.
[57] E. J. Parish and K. Duraisamy. A paradigm for data-driven predictive

modeling using field inversion and machine learning. Journal of
Computational Physics, 305:758–774, 2016. → page 15
[58] J. Han, A. Jentzen, and E. Weinan. Solving high-dimensional partial

differential equations using deep learning. Proceedings of the National
Academy of Sciences, 115(34):8505–8510, 2018. → page 15
[59] I. E. Lagaris, A. Likas, and D. I. Fotiadis. Artificial neural networks for

solving ordinary and partial differential equations. IEEE transactions on
neural networks, 9(5):987–1000, 1998.
[60] S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz. Data-driven

discovery of partial differential equations. Science Advances, 3(4):
e1602614, 2017.
[61] L. Lu, P. Jin, and G. E. Karniadakis. Deeponet: Learning nonlinear

operators for identifying differential equations based on the universal
approximation theorem of operators. arXiv preprint arXiv:1910.03193,
2019. → page 15
[62] J. Willard, X. Jia, S. Xu, M. Steinbach, and V. Kumar. Integrating

physics-based modeling with machine learning: A survey. arXiv preprint
arXiv:2003.04919, 2020. → page 15
[63] R. Wang. Physics-guided deep learning for dynamical systems: A survey.

arXiv preprint arXiv:2107.01272, 2021.
136
[64] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. W. Battaglia. Learning
mesh-based simulation with graph networks. arXiv preprint
arXiv:2010.03409, 2020. → pages 16, 130
[65] R. Wang, K. Kashinath, M. Mustafa, A. Albert, and R. Yu. Towards
physics-informed deep learning for turbulent flow prediction. In
Proceedings of the 26th ACM SIGKDD International Conference on
Knowledge Discovery & Data Mining, pages 1457–1466, 2020. → page 15
[66] A. Karpatne, W. Watkins, J. Read, and V. Kumar. Physics-guided neural
networks (pgnn): An application in lake temperature modeling. arXiv
preprint arXiv:1710.11431, 2, 2017. → page 15
[67] M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural
networks: A deep learning framework for solving forward and inverse
problems involving nonlinear partial differential equations. Journal of
[68] Y. Zhu, N. Zabaras, P. Koutsourelakis, and P. Perdikaris.
Physics-constrained deep learning for high-dimensional surrogate
modeling and uncertainty quantification without labeled data. Journal of
[69] N. B. Erichson, M. Muehlebach, and M. W. Mahoney. Physics-informed
autoencoders for lyapunov-stable fluid flow prediction. arXiv preprint
arXiv:1905.10866, 2019. → page 15
[70] N. Geneva and N. Zabaras. Modeling the dynamics of pde systems with
physics-constrained deep auto-regressive networks. Journal of
Computational Physics, 403:109056, 2020. → page 15
[71] W. Mallik, R. K. Jaiman, and J. Jelovica. Kinematically consistent
recurrent neural networks for learning inverse problems in wave
propagation. arxiv.org/pdf/2110.03903.pdf, 2021. → page 15
[72] J. Wang, J. Wu, and H. Xiao. Physics-informed machine learning approach
for reconstructing reynolds stress modeling discrepancies based on dns
data. Physical Review Fluids, 2(3):034603, 2017. → page 15
[73] A. Daw, R. Q. Thomas, C. C. Carey, J. S. Read, A. P. Appling, and
A. Karpatne. Physics-guided architecture (pga) of neural networks for
quantifying uncertainty in lake temperature modeling. In Proceedings of
the 2020 SIAM International Conference on Data Mining, pages 532–540.
SIAM, 2020. → page 15
137
[74] B. Chang, M. Chen, E. Haber, and E. H. Chi. Antisymmetricrnn: A
dynamical system view on recurrent neural networks. arXiv preprint
arXiv:1902.09689, 2019. → page 15
[75] N. Muralidhar, M. R. Islam, M. Marwah, A. Karpatne, and

N. Ramakrishnan. Incorporating prior domain knowledge into deep neural
networks. In 2018 IEEE International Conference on Big Data (Big Data),
pages 36–45. IEEE, 2018. → page 15
[76] L. Ruthotto and E. Haber. Deep neural networks motivated by partial

differential equations. Journal of Mathematical Imaging and Vision, pages
1–13, 2019. → page 15
[77] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart,

and A. Anandkumar. Fourier neural operator for parametric partial
differential equations. arXiv preprint arXiv:2010.08895, 2020. → page 15
[78] X. Guo, W. Li, and F. Iorio. Convolutional neural networks for steady flow
approximation. In Proceedings of the 22nd ACM SIGKDD international
conference on knowledge discovery and data mining, pages 481–490, 2016.
→ pages 15, 18
[79] S. Lee and D. You. Data-driven prediction of unsteady flow over a circular
cylinder using deep learning. Journal of Fluid Mechanics, 879:217–254,
2019. → pages 15, 18
[80] R. Han, Y. Wang, Y. Zhang, and G. Chen. A novel spatial-temporal

prediction method for unsteady wake flows based on hybrid deep neural
network. Physics of Fluids, 31(12):127101, 2019. → page 15
[81] F. Ogoke, K. Meidani, A. Hashemi, and A. B. Farimani. Graph

convolutional neural networks for body force prediction. arXiv preprint
arXiv:2012.02232, 2020. → page 16
[82] A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and

P. Battaglia. Learning to simulate complex physics with graph networks. In
International Conference on Machine Learning, pages 8459–8468. PMLR,
2020. → page 16
[83] L. Sirovich. Turbulence and the dynamics of coherent structures. i.

coherent structures. Quarterly of applied mathematics, 45(3):561–571,
1987. → page 16
138
[84] P. J. Schmid. Dynamic mode decomposition of numerical and experimental
data. Journal of fluid mechanics, 656:5–28, 2010. → page 16
[85] J. R. Singler. Balanced pod for model reduction of linear pde systems:
convergence theory. Numerische Mathematik, 121(1):127–164, 2012. →
page 16
[86] S. Peitz and S. Klus. Koopman operator-based model reduction for

switched-system control of pdes. Automatica, 106:184–191, 2019. → page
16
[87] T. P. Miyanawala and R. K. Jaiman. Decomposition of wake dynamics in

fluid–structure interaction via low-dimensional models. Journal of Fluid
Mechanics, 867:723–764, 2019. → pages 16, 22, 30, 32, 64, 65
[88] P. Baldi and K. Hornik. Neural networks and principal component analysis:
Learning from examples without local minima. Neural networks, 2(1):
53–58, 1989. → page 17
[89] E. Plaut. From principal subspaces to principal components with linear

autoencoders. arXiv preprint arXiv:1804.10253, 2018. → page 17
[90] D. Park, Y. Hoshi, and C. C. Kemp. A multimodal anomaly detector for

robot-assisted feeding using an lstm-based variational autoencoder. IEEE
Robotics and Automation Letters, 3(3):1544–1551, 2018. → page 17
[91] M. Ma, C. Sun, and X. Chen. Deep coupling autoencoder for fault
diagnosis with multimodal sensory data. IEEE Transactions on Industrial
Informatics, 14(3):1137–1145, 2018. → page 17
[92] J. Yu, C. Hong, Y. Rui, and D. Tao. Multitask autoencoder model for
recovering human poses. IEEE Transactions on Industrial Electronics, 65
(6):5060–5068, 2017. → page 17
[93] G. Dong, G. Liao, H. Liu, and G. Kuang. A review of the autoencoder and
its variants: A comparative perspective from target recognition in
synthetic-aperture radar images. IEEE Geoscience and Remote Sensing
Magazine, 6(3):44–68, 2018. → page 17
[94] S. Fresca and A. Manzoni. Real-time simulation of parameter-dependent

fluid flows through deep learning-based reduced order models. arXiv
preprint arXiv:2106.05722, 2021. → pages 17, 18
139
[95] A. T. Mohan and D. V. Gaitonde. A deep learning based approach to
reduced order modeling for turbulent flow control using lstm neural
networks. arXiv preprint arXiv:1804.09269, 2018. → page 17
[96] F. Pichi, F. Ballarin, G. Rozza, and J. S. Hesthaven. An artificial neural

network approach to bifurcating phenomena in computational fluid
dynamics. arXiv preprint arXiv:2109.10765, 2021. → page 17
[97] Q. Wang, J. S. Hesthaven, and D. Ray. Non-intrusive reduced order

modeling of unsteady flows using artificial neural networks with
application to a combustion problem. Journal of computational physics,
384:289–307, 2019. → page 17
[98] J. S. Hesthaven and S. Ubbiali. Non-intrusive reduced order modeling of

nonlinear problems using neural networks. Journal of Computational
Physics, 363:55–78, 2018. → page 17
[99] W. Chen, Q. Wang, J. S. Hesthaven, and C. Zhang. Physics-informed

machine learning for reduced-order modeling of nonlinear problems.
Journal of Computational Physics, page 110666, 2021. → page 17
[100] S. Bhatnagar, Y. Afshar, S. Pan, K. Duraisamy, and S. Kaushik. Prediction

of aerodynamic flow fields using convolutional neural networks.
Computational Mechanics, pages 1–21, 2019. → page 18
[101] X. Jin, P. Cheng, W. Chen, and H. Li. Prediction model of velocity field
around circular cylinder over various reynolds numbers by fusion
convolutional neural networks based on pressure on the cylinder. Physics of
Fluids, 30(4):047105, 2018.
[102] M. Eichinger, A. Heinlein, and A. Klawonn. Stationary flow predictions

using convolutional neural networks. In Numerical Mathematics and
Advanced Applications ENUMATH 2019, pages 541–549. Springer, 2021.
[103] T. Murata, K. Fukami, and K. Fukagata. Nonlinear mode decomposition

with convolutional neural networks for fluid dynamics. Journal of Fluid
Mechanics, 882, 2020. → page 18
[104] J. Rabault, J. Kolaas, and A. Jensen. Performing particle image

velocimetry using artificial neural networks: a proof-of-concept.
Measurement Science and Technology, 28(12):125301, 2017. → page 18
140
[105] Y. Lee, H. Yang, and Z. Yin. Piv-dcnn: cascaded deep convolutional neural
networks for particle image velocimetry. Experiments in Fluids, 58(12):
171, 2017. → page 18
[106] S. Chaturantabut and D. C. Sorensen. Discrete empirical interpolation for

nonlinear model reduction. SIAM Journal on Scientific Computing, 32:
2737–2764, 2010. → page 22
[107] S. S. An, T. Kim, and D. L. James. Optimizing cubature for efficient

integration of subspace deformations. ACM Transactions on Graphics,
165, 2008. → pages 22, 130
[108] Y. LeCun, Y. Bengio, et al. Convolutional networks for images, speech,

and time series. The handbook of brain theory and neural networks, 3361
(10):1995, 1995. → page 24
[109] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst.

Geometric deep learning: going beyond euclidean data. IEEE Signal
Processing Magazine, 34(4):18–42, 2017. → pages 25, 130
[110] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural

computation, 9(8):1735–1780, 1997. → page 27
[111] C. Olah. Understanding lstm networks. 2015. → page 28
[112] J. Burkardt, M. Gunzburger, and H. Lee. Pod and cvt-based reduced-order

modeling of navier–stokes flows. Computer Methods in Applied Mechanics
and Engineering, 196(1-3):337–355, 2006. → page 30
[113] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization.

arXiv preprint arXiv:1412.6980, 2014. → pages 32, 37, 77
[114] Scipy reference guide, release 1.4.1.

https:// docs.scipy.org/ doc/ scipy-1.4.1/ scipy-ref-1.4.1.pdf , 2019. → pages
38, 79, 100, 107
[115] P. Alfeld. A trivariate clough—tocher scheme for tetrahedral data.

Computer Aided Geometric Design, 1(2):169–181, 1984. → page 38
[116] G. M. Nielson. A method for interpolating scattered data based upon a

minimum norm network. Mathematics of Computation, 40(161):253–271,
1983. → page 39
141
[117] R. J. Renka, R. L. Renka, and A. K. Cline. A triangle-based c1
interpolation method. The Rocky Mountain journal of mathematics, pages
223–237, 1984. → page 39
[118] B. Liu and R. K. Jaiman. Interaction dynamics of gap flow with

vortex-induced vibration in side-by-side cylinder arrangement. Physics of
Fluids, 28(12):127103, 2016. → page 45
[119] R. Mojgani and M. Balajewicz. Physics-aware registration based

auto-encoder for convection dominated pdes. arXiv preprint
arXiv:2006.15655, 2020. → page 64
[120] T. Taddei. A registration method for model order reduction: data

compression and geometry reduction. SIAM Journal on Scientific
Computing, 42(2):A997–A1027, 2020. → page 64
[121] A. Venkatraman, M. Hebert, and J. A. Bagnell. Improving multi-step

prediction of learned time series models. In Twenty-Ninth AAAI
Conference on Artificial Intelligence, 2015. → page 77
[122] K. Antczak. Deep recurrent neural networks for ecg signal denoising.
arXiv preprint arXiv:1807.11551, 2018. → page 88
[123] A. Maas, Q. V. Le, T. M. O’neil, O. Vinyals, P. Nguyen, and A. Y. Ng.

Recurrent neural networks for noise reduction in robust asr. 2012. → page
88
[124] S. Han, Z Meng, X. Zhang, and Y. Yan. Hybrid deep recurrent neural
networks for noise reduction of mems-imu with static and dynamic
conditions. Micromachines, 12(2):214, 2021. → page 88
[125] M. C. Mozer, D. Kazakov, and R. V. Lindsey. State-denoised recurrent

neural networks. arXiv preprint arXiv:1805.08394, 2018. → page 88
[126] T. A. Johnson and V. C. Patel. Flow past a sphere up to a reynolds number

of 300. Journal of Fluid Mechanics, 378:19–70, 1999. → page 106
[127] Y. Tay, M. Dehghani, D. Bahri, and D. Metzler. Efficient transformers: A

survey. arXiv preprint arXiv:2009.06732, 2020. → page 130
142

Ubc 2022 May Gupta Rachit

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ubc 2022 May Gupta Rachit

Uploaded by

Copyright:

Available Formats

Deep Learning-based Reduced Order Modeling for

Unsteady Flow Dynamics and Fluid-Structure Interaction

B.Tech., Delhi Technological University, 2019

A THESIS SUBMITTED IN PARTIAL FULFILLMENT

Master of Applied Science

THE FACULTY OF GRADUATE AND POSTDOCTORAL

The University of British Columbia

© Rachit Gupta, 2022

Deep Learning-based Reduced Order Modeling for Unsteady Flow Dy-

This work presents data-driven predictions of nonlinear dynamical systems involv-

Large-scale simulations of fluid-structure systems have been possible using high-

This M.A.Sc. thesis entitled “Deep Learning-based Reduced Order Modeling

Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

List of symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

4 Low-dimensional Analysis via Deep Learning . . . . . . . . . . . . . 44

5 Partitioned Deep Learning for Fluid-Structure Interaction . . . . . 68

6 Three-dimensional Deep Learning for Parametric Unsteady Flows . 99

7 Conclusions and Recommendations . . . . . . . . . . . . . . . . . . 125

Table 5.1 An elastically-mounted circular cylinder undergoing vortex-induced

Figure 1.1 Schematic of an autonomous digital twin and its components

Figure 6.1 Schematic of a 3D mesh-to-mesh field transfer and load recov-

I want to express my sincere and deepest gratitude to my advisor, Prof. Rajeev

reducing carbon/noise pollution caused by shipping in open waters. Advances in

1.1 Deep learning for FSI

serve translational invariance of an image while recurrent neural networks (RNNs)

1.2 Specific motivation

URN due to structure vibration

Marine life and environment

Figure 1.3: Illustration of underwater radiated noise sources due to propeller

reduce propulsion efficiency, as well as introduce the unwelcome risks of longer-

1.3 Objectives and methodology

Field Data Low-order projection

Deep Neural Database ROM approximation

Prediction Add DNN to Database

Flow predictions Parametric forces

Figure 1.4: Schematic of integration framework for high-fidelity simulations 5

with DL-based reduced-order modeling for data-driven computing.

• Can we construct an efficient DL-ROM for data-driven time series predic-

• Can we generalize the effects of parameters (e.g., Reynolds number) for

• Data-driven study on unsteady flow predictions for complex wake interfer-

• Generalized partitioned and modular DL-ROM approach to couple the fluid

• Efficient time series predictions of FSI for a freely vibrating structure in

• Effective learning and inference capabilities of DL-ROM for unsteady flow

• Generalized 3D data-driven DL-ROM for parametric flow and force predic-

1.5 Organization of thesis

2.1 Overview of high-fidelity simulations

2.2 Physics-based deep learning

2.3 Deep learning-based reduced order modeling

In this chapter, we start by describing the full-order equations of a coupled fluid-

3.1 Full-order modeling for fluid-structure dynamics

uf (t) = us (t) , (3.4)

3.2 Reduced-order modeling

F(z) = C + Bz + F′ (z). (3.7)

3.2.1 Projection-based reduced-order modeling

Here, the reduced-order space is constructed by defining a choice of modes V ∈

Snapshots Convolutional autoencoder Long short-term memory nets

Figure 3.1: Illustration of the projection and autoencoder (AE) based

3.2.2 Autoencoder-based reduced-order modeling

θ AE = argminθ AE [E(Z, F (Z; θ AE ))]. (3.9)

3.3 Deep learning methods

3.3.1 Convolutional neural networks

Figure 3.2: An abstract representation of a convolutional neural network