International Communications in Heat and Mass Transfer 132 (2022) 105890

Contents lists available at ScienceDirect

International Communications in Heat and Mass Transfer

journal homepage: www.elsevier.com/locate/ichmt

On the application of physics informed neural networks (PINN) to solve

boundary layer thermal-fluid problems
Hassan Bararnia a, 1, Mehdi Esmaeilpour b, *, 1
a
Independent researcher
b
Department of Mechanical Engineering, College of Engineering and Computer Sciences, Marshall University, Huntington, WV 25755, USA

A R T I C L E I N F O A B S T R A C T

Keywords: Deep neural network is a powerful technique in discovering the hidden physics behind the transport phenomena
Physics-informed neural networks through big-data training. In this study, the application of physic-informed neural networks is extended to solve
Boundary layer flow viscous and thermal boundary layer problems. Three benchmark problems including Blasius-Pohlhausen, Falk­
Heat transfer
ner-Skan, and Natural convection are selected to investigate the effects of nonlinearity of the equations and
Machine learning, nonlinear ODEs
unbounded boundary conditions on adjusting the network structure’s width and depth, leading to reasonable
solutions. TensorFlow is used to build and train the models, and the resulted predictions are compared with those
obtained by finite difference technique with Richardson extrapolation. The results reveal that the Prandtl number
in the heat equation is a key factor which its value drastically changes the required number of neurons and layers
to achieve the desired solutions. Also, setting the unbounded boundary at a higher distance from the origin
demands an adequate number of layers and correspondingly neurons to deal with the infinity boundary con­
dition. Finally, trained models are successfully applied to unseen data to evaluate the boundary layer thicknesses.

1. Introduction
Machine learning is a promising science in statistical methods with a
diverse application in forecasting, pattern/speech recognition, optimi­
zation, time-series analysis, and image processing [19,33,41]. A neural
network is a branch of machine learning concept that mimics the human
brain’s biological computing algorithm. The neural network field has
gained significant interest in various areas of science. The applications
include but are not limited to computational material and chemistry
science [2,18], the pharmaceutical industry [9], biomedical engineering
[25], forecasting sciences [38], robotics [42], aerospace, and mechani­
cal engineering [23,37]. The motivation behind using neural networks
emanates from providing a universal approximation for continuous
functions even with a single hidden layer in their architecture [21] for
the case of simple functions. The complexity of the functions brings the
necessity of adding more layers (i.e., “deep learning”) and neurons (i.e.,
sufficiently wide layers) to the network [11,15]. For instance, in fluid
mechanics, the high-dimensionality and nonlinearity of the problems
present a significant challenge, especially in modeling and extracting the
fluid features from big data [8]. Therefore, several efforts have been
dedicated to employ neural networks in the fluid mechanic and heat
transfer field in recent years. It is interesting to note that the application
of shallow networks (networks with zero or one hidden layer) in fluid
mechanics emerged in the early 1990s [52] to track the low-density
particles’ velocity through analyzing images (PIV), which further
improved by multi-layer networks [20]. In the context of computational
fluid dynamics (CFD), the multi spatiotemporal nature of turbulent flow
and stochastic formation of coherent structures drew the attention of
scientists to reduced-order models aimed at capturing the dominant
physics of a phenomenon of interest. Milano and Koumoutsakos [34]
were among the first who explored the capability of a multi-layer
network for constructing the near-wall turbulent flow. Further in­
vestigations have been exploring neural networks’ applications in tur­
bulent modelings [32,49,51]. Neural networks have also been used in
classification problems attributed to the identification of different re­
gimes in multiphase flow (bubbly, slug, plug, etc.) [1,54] or detecting
the various modes of vortexes forming behind an airfoil [12].
Neural networks have been employed to obtain the partial differ­
ential equations’ solutions by reducing the problem to an optimization
process rather than numerically solving the equations. Among the early
efforts in this topic, one can be referred to Dissanayake and Phan-Thien
[13] or Lagaris et al. [27]. In the method proposed by Lagaris et al. [27],

* Corresponding author.
E-mail address: esmaeilpour@marshall.edu (M. Esmaeilpour).
1
These authors contributed equally to this work.

https://doi.org/10.1016/j.icheatmasstransfer.2022.105890

Available online 14 February 2022

0735-1933/© 2022 Elsevier Ltd. All rights reserved.
H. Bararnia and M. Esmaeilpour
a trial solution is assumed to satisfy the ordinary/partial differential
equation. The trial solution is written as the sum of two terms, such that
the first term is responsible for dealing only with the boundary/initial
conditions while the second term is a neural network system that does
not contribute to satisfying initial/boundary conditions yt = A(x) + F(x,
N(x, p)). Here, x indicates the independent variable(s) of the equation.
The parameter p corresponds to biases, and the network’s weights must
be adjusted to minimize the residuals. Although the procedure of the
solution seems straightforward and it has been shown that even a
shallow network is capable of solving the equations, finding the trial
solution might be arduous, particularly in time-dependent and multi-
coordinate problems with complicated geometries. Recently, re­
searchers at Brown University [46–48] developed a deep neural network
method for solving nonlinear partial differential equations (PDEs)
without requiring a trial function. The method is based on introducing a
general neural network solution that requires satisfying both initial/
boundary conditions as well as the corresponding PDE. Therefore, the
necessity of decomposing the answer into two terms has been elimi­
nated. The motivation stems from the fact that contrary to the conven­
tional supervised machine learning problems, which require a given
output labeled data, here in the context of solving equations, the actual
outputs, which are the solution of the equations, are unknown. Hence, a
valid question that can be posed here is how the corresponding loss
function can be defined in that scenario? The method is termed a
“Physic-Informed Neural Network (PINN)” since the dilemma of
defining a proper loss function is resolved by prior knowledge of the
underlying physics of the problem elucidated by the governed equa­
tions. In other words, the ultimate result of the output layer is expected
to satisfy the governing equations for every node located inside the
domain (collocated nodes) in addition to boundary nodes. Therefore, the
loss function is defined based on two terms: 1) the residual value
resulted from substituting the predicted output of the neural network in
the corresponding equations, and 2) the discrepancy between the ex­
pected value and the estimated one at the boundaries. The summation of
these losses as a total loss undergoes a minimizing process by optimizing
the weights. The detail regarding the method is discussed in the
following sections. Since the introduction of the technique, it has been
widely employed in different branches of science. For instance, in the
domain of CFD and heat transfer, Rad et al. [45] successfully simulated a
solidification problem in a two-dimensional domain. Zobeiry and
Humfeld [56] solved a heat conduction problem with a convective
boundary condition and compared their results with the Finite element
method (FEM), and Wang et al. [53] reconstructed the velocity, tem­
perature distributions induced by a natural convection mechanism
within the enclosure. Moreover, Chen et al. [10] showed the application
in inverse problems related to the optics field.
The theory of boundary layer flow is a classical problem in the
thermal-fluid field, having numerous applications in engineering and
industrial processes. For example, determining the boundary layer
thickness (thermal and hydraulic) beside the gradient of flow quantities
(temperature and velocity) enables the engineers to reduce the drag
force on airfoils, turbine blades, and ship hull, [6,14,28,35,44], leading
to a substantial decrease of fuel consumption and improved efficiency.
The celebrated boundary layer theory was first introduced by Ludwig
Prandtl [43] in a paper entitled “On the motion of a fluid with very small
viscosity” in 1904, in which he presented the mathematical basis of
flows for high Reynolds numbers and simplified the two-dimensional
Navier-Stokes equations into the boundary layer equations. This paper
has been considered as the onset of modern fluid mechanics and opened
the gates to understanding fluid motion physics. Later, Blasius [7] pre­
sented a power series solution for the boundary layer equation of flow
over a flat plate. Accordingly, the generalization format of the Prandtl-
Blasius equation to non-zero pressure gradient flow over a fixed and
impermeable wedge was introduced by Falkner and Skan [17]. Another
example of classical boundary layer problems is the natural convection
boundary layer flow over a vertical flat plate which has numerous
2
International Communications in Heat and Mass Transfer 132 (2022) 105890
industrial applications in cooling systems of electronic devices and many
manufacturing processes. In the natural convection scenario, both
viscous and thermal boundary layers develop simultaneously, induced
by the velocity and temperature gradient, respectively, in the presence
of wall and gravity effects. The primary studies on this topic were done
by Schmidt and Beckmann [50] and Ostrach [39], who experimentally
and theoretically studied the free convection flow of air subject to the
gravitational force about a vertical flat plate. Introducing the boundary
layer theory and obtaining a closed-form solution for the governed
equations had always been a challenge. The difficulties lie in two as­
pects: 1) the two dimensional coupled nonlinear partial differential
equations, 2) unbounded boundary conditions (A zero-gradient of
quantities occurring at infinity limit). Although the first issue was suc­
cessfully addressed by introducing the similarity variables and con­
verting the PDEs to nonlinear ODEs in one coordinate, the second
challenge required special techniques to deal with unbounded boundary
conditions. While numerical methods like Runge-Kutta method [22]
could reasonably estimate the infinity value, exact solutions were not
the case. Therefore, finding an approximate or analytical solution has
been the subject of many studies. Most of these techniques rely on
polynomial expansions of the solution, e.g., the Adomian Decomposition
Method [5] and the Homotopy Perturbation Method [16]. In these
methods, the primary step of the solution procedure initiates with
selecting the linear part of the equation as an invertible linear operator;
therefore, a polynomial solution is obtained, and one requires consid­
ering a specific value for infinity since taking the derivative at infinity is
impractical when a function is expanded in polynomial format. On the
other hand, a Pade’ approximation can convert the final polynomial
solution to a ratio that enables us to take the limit when the independent
parameter approaches infinity [3]. Liao [30,31], for the first time,
introduced an analytic technique termed the Homotopy Analysis
Method in which an arbitrary linear operator can be employed, yielding
the solution to take the exponential formats rather than polynomial
expansions. At the same time, its convergence can be checked by
adjusting appropriate auxiliary parameters. As a result, the method
automatically recognizes the final infinity value rather than forcefully
setting an arbitrary infinity value.
In the concept of the neural network, the benchmark problem of
Blasius (viscous flow boundary layer) has been solved [36] by the
method proposed by Lagaris [27] (trial function) or a Hybrid approach
[4]. It should be noted here that the Blasius equation is a single equation
representing only the viscous boundary layer. The main objective of this
work is to explore the application of PINN in solving the broader aspects
of the boundary layer flow equations with unbounded domains, which
to the best of our knowledge, is the first study employing PINN in this
field. The PINN technique will be used to solve a system of coupled
nonlinear thermal and viscous boundary layer equations derived by
similarity solutions for three well-known cases (Blasius-Pohlhausen,
Falkner-Skan, and Natural convection). In addition, the effect of width
and depth of the proposed neural network on correctly estimating the
infinity value will be discussed. Moreover, it will be demonstrated that
how the learning rate as one of the hyper-parameter can result in un­
reasonable results when it is not set sufficiently low. Unlike the Blasius
equation, the other two problems involve some non-dimensional pa­
rameters whose values highly affect the problem’s nonlinearity. Finally,
a thorough investigation of the network architecture will be performed
to address the challenge of nonlinearity to find results that are reason­
ably in good agreement with those obtained by the numerical method.
The paper is structured as follows: section (2) presents the governing
equations corresponding to the three cases. Then, section (3) outlines
the PINN essential elements and the solution procedure, and finally
section (4) details the discussion of the results.
H. Bararnia and M. Esmaeilpour International Communications in Heat and Mass Transfer 132 (2022) 105890

Fig. 1. The Schematic of (A) Blasius-Pohlhausen problem, (B) Falkner-Skan problem, and (C) Natural convective flow over a vertical flat plate. δ(x) and δt(x) indicate
viscous and thermal boundary layers, respectively. Note that other parameters in the schematics are already introduced in the corresponding equations.

2. Mathematical boundary layer models √̅̅̅̅̅̅̅̅̅

U∞ ν
ψ (x, y) = x f (η), U∞ = ax (7)
2.1. Convective flow over the flat plate: Blasius-Pohlhausen equations x
Using the similarity variable η and non-dimensional form of the
Consider a laminar, steady, incompressible cold-flow with uniform temperature θ(η) as:
velocity of U∞ and temperature of T∞ over a heated flat plate with √̅̅̅̅̅̅̅
surface temperature of Ts. A reference frame with x-axis tangential and η(x, y) = y
U∞
(8)
y-axis normal to the plate is considered (Fig. 1A). It is assumed that no νx
significant gradients of pressure exist in x-direction along the plate, and
T − T∞
Boussinesq’s approximation is taken within the boundary layer. Under θ(η) = (9)
the boundary layer and constant property assumptions, the simplified Ts − T∞
Navier-Stokes and energy equations based on all assumptions are given
the momentum and energy partial differential equations (Eqs. (2)–(3))
as:
are reduced into a set of coupled ordinary differential equations as:
∂u ∂v
+ =0 (1) 1 ′
(10)
′ ′
∂x ∂y f ′ + ff ′ = 0
2

∂u ∂u ∂2 u 1
(2) (11)
′ ′
u + v = ν 2 + gβ(T − T∞ ) θ ′ + Prf θ = 0
∂x ∂y ∂y 2

∂T ∂T ∂2 T where f is a function of the similarity variable (η), and Pr is the Prandtl

u +v =α 2 (3) number. The f can be related to the u velocity by:
∂x ∂y ∂y
(12)
′
f = u/U∞
where u and v are the x and y components of the velocity, respectively, ν
is the kinematic viscosity of the fluid, g is the acceleration due to gravity, The corresponding boundary conditions can also be obtained from
β is the coefficient of thermal expansion, T is the temperature of the similarity variable η and non-dimensional temperature θ as:
fluid, and α is the thermal diffusivity of the fluid. Thus, the velocity and
(13)
′ ′

temperature boundary conditions can be expressed as: f (0) = 0, f (0) = 0, f (∞) = 1

u(x, 0) = 0, v(x, 0) = 0, u(x, ∞) = U∞ (4) θ(0) = 1, θ(∞) = 0 (14)

T(x, 0) = Ts , T(x, ∞) = T∞ , T(0, y) = T∞ (5) The governing Eqs. (10)− (11) along with boundary conditions (13)–
(14) define the two-dimensional convective fluid flow over a flat plate,
The first two boundary conditions in Eq. (4) refer to the no-slip ve­ which is equivalent to classical Blasius-Pohlhausen equations for viscous
locity condition, while the last boundary condition in Eq. (4) refers to flow. The velocity in the y-direction and local friction on the flat plate
the constant free stream velocity outside the boundary layer. According can be calculated as:
to Eqs. (2)-(3), the momentum and energy equations are coupled. √̅̅̅̅̅̅̅̅̅
However, the buoyancy force presented in the momentum equation can 1 νU∞ ′
v= − [ηf (η) − f (η) ] (15)
be neglected if a pressure gradient is perpendicular to the gravitational 2 x
force. Therefore, in the case of the forced convection over a horizontal √̅̅̅̅̅̅̅
( )⃒
flat plate, the solution of the momentum equation is decoupled from the ∂u ⃒⃒ U 3∞ ′ ′
energy equation. In contrast, the energy equation solution is still linked τ=μ =μ f (0) = − μω(0) (16)
∂y ⃒y=0 νx
to the momentum solution [24]. Introducing the stream function ψ (x, y)
as: where ω is the vorticity generated by the boundary layer flow. From the
∂ψ (x, y) ∂ψ (x, y) expression for the temperature, the heat flux into the fluid, qw, the heat
u= ,v = − (6) transfer coefficient from the wall to the liquid, h, and the local Nusselt
∂y ∂x
number, Nux, can be expressed as:
can automatically satisfy the continuity Eq. (1). The stream function √̅̅̅̅̅̅̅
∂T U∞ ′
ψ (x, y) is defined as: qw = − kf |y=0 = (Ts − T∞ ) θ (0) (17)
∂y νx

3
H. Bararnia and M. Esmaeilpour International Communications in Heat and Mass Transfer 132 (2022) 105890

√̅̅̅̅̅̅̅
qw U∞ ′ 2.3. Free convection boundary layer: flow over vertical flat plate
h= | = kf θ (0) (18)
Ts − T∞ y=0 νx
Consider the laminar natural convection flow of an incompressible
√̅̅̅̅̅̅̅
hx U∞ ′ viscous fluid over a vertical flat plate. The flow is parallel to the direc­
Nux = = θ (0) = Rex θ′ (0) (19) tion of buoyancy force, and the flat plate has a constant temperature of
kf νx
Ts which is greater than the temperature of the surrounding fluid, T∞. It
2.2. Convective flow over a wedge: Falkner-Skan equation is assumed that the flow is two-dimensional (Fig. 1C). The mass, mo­
mentum, and energy equations for such a fluid flow can be expressed as:
Consider the viscous fluid flow with constant and uniform free ∂u ∂v
stream velocity, U∞, and temperature, T∞, over a wedge. The wall of the + =0 (33)
∂x ∂y
wedge has constant temperature of Ts, greater than the free stream’s
temperature. It is assumed that the flow in the laminar boundary layer is ∂u ∂u ∂2 u
two-dimensional (Fig. 1B). Assuming that the temperature changes due u + v = ν 2 + gx β(T − T∞ ) (34)
∂x ∂y ∂y
to viscous dissipation are small, the simplified Navier-Stokes and energy
equations may be expressed as: ∂T ∂T ∂2 T
u +v =α 2 (35)
∂u ∂v ∂x ∂y ∂y
+ =0 (20)
∂x ∂y The boundary conditions for such a flow are given by:

∂u ∂u dU ∂2 u u(x, 0) = v(x, 0) = 0, u(x, ∞) = 0, u(0, y) = 0 (36)

u +v = U +ν 2 (21)
∂x ∂y dx ∂y
T(x, 0) = Ts , T(x, ∞) = T∞ , T(0, y) = T∞ (37)
2
∂T ∂T ∂T where u and v are the velocity in the x and y direction, respectively, g is
u +v =α 2 (22)
∂x ∂y ∂y the acceleration due to gravity, and β is the coefficient of thermal
expansion. Introducing the free convection similarity variable, η, stream
where U is the reference velocity at the edge of the boundary layer and a
function, ψ (x, y), and non-dimensional form of the temperature θ(η, Pr)
function of x, the boundary conditions for such a fluid flow are given by:
[26,39] as:
u(x, 0) = v(x, 0) = 0, u(x, ∞) = U(x) = U∞ (x/L)m , u(0, y) = U∞ (23) [ ]1/4
gβ(Ts − T∞ )x3 y
η(x, y) = (38)
T(x, 0) = Ts , T(x, ∞) = T∞ , T(0, y) = T∞ (24) 4ν2 x

[ ]1/4
where m is the Falkner-Skan power-law parameter and L is the length of gβ(Ts − T∞ )x3
the wedge. Introducing the similarity variable η, f(η), and non- ψ (x, y) = 4νf (η) (39)
4ν2
dimensional form of the temperature θ(η) as:
√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ T − T∞
(m + 1)U∞ y θ(η, Pr) = (40)
η(x, y) = (25) Ts − T∞
2νLm x(1− m)/2
√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ and substituting Eqs. (38)–(40) into the mass, momentum, and energy
(m + 1)Lm ψ Eqs. (33)–(35) will result in a set of coupled ordinary differential
f (η) = (26)
2νU∞ x(1+m)/2 equations as:

(41)
′ ′ ′ ′
f ′ − 2f 2 + 3ff ′ + θ = 0
T − Ts
θ(η) = (27)
T∞ − Ts
(42)
′ ′
θ ′ + 3Prf θ = 0
and substituting relations (25)–(27) into the mass, momentum, and The corresponding boundary conditions can also be obtained from η,
energy Eqs. (20)− (22) will result in a set of coupled ordinary differential f, θ as:
equations as:
(43)
′ ′
( ′ )
f (0) = 0, f (0) = 0, f (∞) = 0
(28)
′ ′ ′
f ′ + ff ′ + β 1 − f 2 = 0
θ(0) = 1, θ(∞) = 0 (44)
(29)
′ ′
θ ′ + Prf θ = 0
The governing Eqs. (41)–(42) along with boundary conditions (43)–
It should be noted that m and β are related through the following (44) define the two-dimensional free convection flow over a vertical flat
equation: plate which is another example of boundary layer flow problem. From
2m the expression for the temperature, the shear stress on the plate and the
β= (30) local Nusselt number can be expressed as [26]:
m+1
√̅̅̅ [ ] √̅̅̅
The corresponding boundary conditions can also be obtained as: ∂u
τw = μ |y=0 = μν 2
2 gβ(Ts − T∞ )x3 34 ′ 2 3
f ′(0) = μν 2 Gr4x f ′ ′(0) (45)
∂y x ν2 x
(31)
′ ′
f (0) = 0, f (0) = 0, f (∞) = 1
[ ]
hx 1 ′ gβ(Ts − T∞ )x3 14 1
θ(0) = 1, θ(∞) = 0 (32) Nux = = − √̅̅̅θ (0) = − √̅̅̅Gr1/4 ′
x θ (0) (46)
kf 2 ν2
2
The governing Eqs. (28)–(29) along with the boundary conditions
(31)− (32) define the two-dimensional convective fluid flow over a where Grx, is the dimensionless Grashof number.
wedge which is equivalent to the classical Falkner-Skan boundary layer
problem.

4
H. Bararnia and M. Esmaeilpour
Fig. 2. The sketch of essential elements in feedforward neural network, (A) discretization of the one-dimensional domain, (B) a feedforward neural network
structure, the corresponding computational algorithm and important parameters used to solve the ODEs.
3. Physics-informed neural networks (PINN)
The main structure of a neural network consists of a series of parallel-
distributed layers termed input, hidden, and output layers organized in a
row receiving the input data and producing the output results. We start
elucidating the essential steps of applying PINN to solve the aforemen­
tioned equations by dividing the one-dimensional domain in direction
(η) (Fig. 2A) into discrete points (nodes). These nodes represent the in­
dependent variable of the equations (input data) varying from zero to
η∞. The major difference between PINN and the method proposed by
Lagaris et al. [27] is that in the latter method, before proceeding to solve
the equation, one needs to suggest a trial function to satisfy the
boundary conditions and a second term that only satisfies the equation
(trained by network) with no contribution in boundary or initial con­
ditions. If a trial function is chosen such that the quantity or its deriv­
ative yield to zero or one when the independent parameter (η)
approaches infinity (∞) (depending on the boundary conditions), then
the whole solution is likely to be obtained. Finding these two terms
before solving the equations is noticeably challenging, especially in
scenarios with complicated boundary conditions. On the other side, in
PINN, there is no such a requirement, and the network trains the whole
solution by considering penalties to satisfy both boundary/initial con­
ditions and the equation. Therefore, in PINN, we need to consider a
finite value for the (∞). In the following section, we discuss the critical
point that should be taken regarding the reasonable guess made for
estimating (∞) in more detail.
Fig. 2A shows two types of nodes, red colored (collocated nodes) and
blue colored (boundary nodes) ones. Note that we are dealing with
steady-state equations. Therefore, the only boundary conditions
required to close the solutions are Dirichlet and Neumann boundary
conditions, shown in Fig. 2B by (D) and (N), respectively. The η-axis is
discretized into m equal divisions (ηi, i = 0, 1, 2, ⋯, m) such that ηi = i ×
Δ and Δ = η∞/m. Unlike the conventional process of node generation in
PINN, we do not generate the nodes randomly. There are m + 1 nodes
along η-axis. The first and the last nodes are labeled as η0 and ηm which
are considered as boundary nodes where the boundary conditions at
zero and infinity are applied while the nodes in between, ηi, where i = 1,
2, ⋯, m − 1, are collocated nodes distributed along η-axis and are
responsible for satisfying the ODEs. The leftmost layer (single input
neuron in Fig. 2B) is passive and responsible for holding the raw input
data (ηi).
The next step is to build a function that maps the input data xi = ηi to
[ ]
the output data ̂y i = fi (η) , θi (η) which are the solutions of ODEs. The
function ̂y i = F (xi , w, b) consists of some learning parameters termed
weights (w) and biases (b) which required to be trained to result in an
acceptable function approximation, such that ̂ y i (estimated value to
5
International Communications in Heat and Mass Transfer 132 (2022) 105890
each xi), reasonably predict the desired value (yi) (sample output data).
A loss function quantifies the difference between (yi) and (̂
y i ), in a mean
squared error format, defined as:
[ ]/
∑
i=m
Loss = (yi − ̂y i )2 m (47)
i=0
The function F is a series of computations performed on all neurons
constructed the network. Before elucidating how the function works, we
should take a look at Fig. 2B, which shows a network architecture in
which the input (ηi) is fed to the network. The neural network sketched
in Fig. 2B depicts that the first layer has only one neuron which stores
the input data (samples). After that, there are three hidden layers with
an arbitrary number of neurons (just the first two nodes and the last
node were shown for the sake of simplicity, and the rest showed by dots).
Finally, the output layer has two nodes. The number of neurons at the
output layer is according to the number of unknown functions which one
intends to solve. The network is constructed by layers (k = 0, 1, 2, ⋯, l),
and each layer involves several neurons (j = 1, 2, ⋯, n). The neurons are
fully connected via connections (links: black arrows). The role of a link is
to transfer the information between two neurons. To better understand
the essential steps and computations performed by a single neuron, a
representative neuron is shown in Fig. 2B (inset). The links from the
neurons of the previous layer (k − 1) showed by green arrows contain
the weighted input data. A weight is assigned to each link (wi, jk), which
dampens or amplifies the data coming from the source node (j) before
entering the destination node (representative node (i)). Thereby, the
weights determine the importance of the signal sent to the node or, in
other words, how much each node contributes to the output result of the
representative node. Then, to consider the contribution of all the nodes
on the target node (i), the sum of weighted inputs is calculated, and
finally, a bias is added to the resulted sum:
( )
∑j=n
zi = bi + j=1
a k
j .wi.j (48)
The bias term (bi) is an extra neuron with a value of one, added to the
input and hidden layers only and not the output layer, and its role is to
decrease the bias of estimated values compared to the desired ones. In
Eq. (48), zi might be thought of as a conventional regression analysis
expressed by f(x) = αx + β, such that α and β are analogous to weights
and biases terms in neural network computations. The bias neuron in a
layer (e.g., k − 1) is linked to all the neurons of the next layer (k) except
its bias neuron. The computed value (zi), then passes through an acti­
vation function to be restricted in a specific range. The function used in
this study is the hyperbolic tangent activation function:
Output (i) = tanh(zi ) (49)
H. Bararnia and M. Esmaeilpour
Fig. 3. Results corresponding to the Case (I): convective flow over a flat plate. The estimated results, obtained by PINN (300, 6) for (A) the flow field (Eq. (10)), and
(B) the temperature and temperature gradient distribution for different Pr numbers (Eq. (11)). (C) presents the effect of learning rate on decreasing the loss function
during the training process, and (D) highlights the role of the width of the network on addressing the extended infinity boundary condition.
The information is processed in a forward direction (connections
between neurons do not make a loop), and at each neuron (unit), the
computations are performed to obtain the output (zi). The procedure
continues layer by layer (Forward pass) until the last layer, which does
not use any activation function, and the value of zi obtained at each node
in this layer is directly counted as the estimated solution (̂
y i ).
Since the procedure initiated by the guessed values of weights and
biases, it is expected to see a significant discrepancy between ̂ y i and yi.
The resulted loss function needs to be minimized, which is impossible
unless the values of weights and biases should be tuned. Finding the
weights and biases which leads the loss function close to zero is called an
optimization process. The essence of the optimization process is to
determine the gradient of the loss function with respect to any weights
and biases of the network, which is accomplished through the chain rule
and a backpropagation algorithm. We used the Adaptive Moment
(Adam) optimizer to tune the weights and biases such that the loss
function is decreased. The iterative process (forward-backward cycles
termed iterations) continues until the loss function meets the pre­
determined criteria of convergence. Once the training process ends, the
model can predict the unseen samples via a single forward pass step.
Note that the expected output (yi) is not available. Therefore, how a
loss function can be defined? Raissi et al. [46–48]2 addressed this
2
https://github.com/maziarraissi/PINNs.
6
International Communications in Heat and Mass Transfer 132 (2022) 105890
question by proposing a loss function based on the ODEs. The logic
[ ]
behind this definition is that the output data ̂ y i = fi (η) , θi (η) are sup­
posed to satisfy the ODEs. Therefore, loss function is defined based on
the concept of how well (̂ y i ) is likely to satisfy the ODEs. Unlike Lagaris
et al. [27], in their proposed method, estimated network output is
responsible for both satisfying ODEs and meeting the corresponding
boundary conditions. Accordingly, the total loss function for solving an
ODE is divided into: 1) the loss function attributed to the boundary
conditions (Neumann and Dirichlet), and 2) the loss function related to
satisfying the ODE itself. In Fig. 2B, four loss functions are shown. The
first two represent the loss functions related to two coupled ODEs (flow
and thermal) equations (Rf and Rθ). The loss function attributed to the
Dirichlet and Neumann boundaries are expressed with (RD and RN),
respectively. The subscripts f and θ indicate whether the corresponding
boundary belongs to the thermal or flow equation. Below, we derived
these loss functions and the overall loss function for the three case
studies that undergo an optimation process afterward.
I. Blasius-Pohlhausen Flow:
[ ]/
∑m− 1 ( ′ ′ 1 ′ )2
′ ′
Rf = f + fi fi (m − 1)
i
i=1 2
H. Bararnia and M. Esmaeilpour International Communications in Heat and Mass Transfer 132 (2022) 105890

Table 1
L 2 - error between the estimated and the numerical solution of f(η), f′ (η) for different number of hidden layers and different number of neurons per layer. The value of
infinity and mesh size are η∞=5.0 and Δ = 0.05, respectively.
Neurons

20 50 100 150

Layers f(η) f (η)

′
f(η) f (η)
′
f(η) f (η)
′
f(η) f′ (η)

2 1.07E-02 1.02E-02 1.07E-02 9.91E-03 1.08E-02 9.96E-03 1.08E-02 9.97E-03

4 1.05E-02 1.04E-02 1.20E-02 1.06E-02 1.16E-02 1.03E-02 1.18E-02 1.05E-02

4. Results and discussions

[ )2 ]/
∑m− 1 ( ′
Rθ = ′ Pr ′
θi + fi θi (m − 1) We start the discussion by analysis of Blasius flow (Eq. (10)). The
i=1 2 ODEs describing the problem were given in section 2 (Case I). As can be
seen, the flow equation does not depend on the temperature distribu­
( ′ )2 ( ′ )2
R Df : (f0 )2 ; R Nf : f0 + fm − 1 ; R Dθ : (θ0 − 1)2 + (θm )2 (50) tion. On the other hand, the heat equation is coupled with the flow field.
Eq. (10) is a good starting point to explore the role of network structure
(width and depth) and the hyperparameter (λ) since it does not contain
II. Falkner-Skan Wedge Flow: any other external parameters such as β or Pr. The solution procedure is
[
∑m− 1 ( ′ ′ (
]/ initiated by constructing a neural network with twenty neurons (width),
′ 2 ) )2
two layers (depth), and by considering η∞=5.0. The learning rate for the
′ ′′
Rf = i=1
fi + fi fi + β 1 − fi (m − 1)
Adam optimizer is set as λ = 10− 4. The corresponding equation is also
solved by finite difference technique with Richardson extrapolation to
evaluate the accuracy of the estimated results by measuring L 2 - error. It
is observed that the PINN results (f, f′ , f") are in good agreement with
[ ]/ those obtained by the numerical method (shown in Fig. 3A). Although
∑m− 1 ( ′ ′ )2 this set of width (w) and depth (D) (20,2) is able to estimate the expected
Rθ = θi′ + Prf i θi (m − 1)
i=1
trends; however, the corresponding L 2 - error (tabulated in Table 1) is
not sufficiently low.
( ′ )2 ( ′ )2
R Df : (f0 )2 ; R Nf : f0 + fm − 1 ; R Dθ : (θ0 − 1)2 + (θm )2 (51) The first approach to achieve higher accuracy is to increase the width
of the network while the number of layers is fixed. Table 1 shows that
increasing the number of neurons to 50, 100, or even more does not
III. Natural Convection: improve the results. Also, doubling the layers does not show any
[
∑m− 1 ( ′ ′
]/ improvement and the L 2 - error approximately remains unchanged in
)2
the order of ~10− 2. It should be noted that the loss function decreases to
′ ′2 ′′
Rf = fi − 2f i + 3fi fi + θ i (m − 1)
below 10− 3, that is why the network is able to predict the expected
i=1

[ ]/
∑m− 1 ( ′ ′ )2
Rθ = θi′ + 3Prf i θi (m − 1)
i=1
( ′ )2 ( ′ )2
R Df : (f0 )2 ; R Nf : f0 + fm ; R Dθ : (θ0 − 1)2 + (θm )2
The total loss for each case is the summation of all these losses:
Loss = R f + R θ + R Df + R Nf + R Dθ
Therefore, each training point should meet a specific condition
depending on whether it is located on the boundaries or inside the
domain. Recall that minimizing the total loss function to find the proper
weight coefficients requires a forward pass followed by a back­
propagation step termed an iteration (epoch). The loss function (error) is
calculated at each epoch, indicating how well the necessary conditions
are met. Also, all the derivatives that exist in the loss function are
calculated using a method known as automatic differentiation available
in the TensorFlow package.
The optimizer updates the weights and biases during the training to
decrease the overall lost function at each epoch. As a result, a reliable
network shows a decrease in loss value as per epoch. The training con­
tinues until the convergence criterion is achieved (the difference of the
loss value between the two successive epochs falls below a threshold, e.
g., 10− 5).
trends (converged solution) but with low accuracy, indicating that the
issue is not related to the network structure. Instead, we observe that
considering the infinity boundary at η∞=5.0 is not a proper guessed
value since the numerical solution yields to one and zero for (f′ and θ),
respectively, when the independent variable η approaches to the value
greater than 5.0, signifying that arbitrary value of (η∞) for boundary
layer problems should be selected carefully to ensure that we are far
beyond the boundary layer thickness, specifically when dealing with
(52)
equations with different boundary layer thicknesses (discussed later).
As the second try towards reaching the higher accuracy, the position
of the guessed value is extended to η∞= 8.0. However, it is found that the
(53)
solution does not converge and the loss function fluctuates around 10− 2
(Fig. 3D), declaring that the ODE is not satisfied by this network struc­
ture (20, 2), resulting in an incorrect result (Fig. 3A- dashed lines). The
predominant inference is that the network structure requires a modifi­
cation with regard to depth and width when the domain’s boundary is
extended and the learning rate parameter is sufficiently low. In this
regard, one of the crucial questions, for which no conclusive deduction
has been proposed yet, is which of these approaches is more beneficial,
increasing the depth or widening the layers [40]. For better under­
standing these effects, the network structure is limited to a fixed number
of layers D = 2 while the number of neurons is gradually increased,
varying from w = 20 to 120. It is found that increasing the width to 30
neurons and above gives rise to a converged solution when the depth is
fixed at D = 2. Hence, increasing the width indeed resolves the problem
mentioned above. However, it might be thought that considering the
units more than 30, in this case, would always result in a converged
solution even if the number of layers is increased. But, this is not always
the case. In other words, the question that can be posed here is whether

7
H. Bararnia and M. Esmaeilpour International Communications in Heat and Mass Transfer 132 (2022) 105890

Table 2
L 2 - error between the estimated and the numerical solutions of f(η), f′ (η) for different number of hidden layers and different range of neurons per
layer. The value of infinity and mesh size are considered as η∞=8.0 and Δ = 0.05, respectively. Note that (− ) declares a range of examined neurons
at each coloumn and the tabulated data are attributed to the number of neurons written in the parenthesis. Also, there are two exceptional sets (30,
4, 6, 130) such that the latter does not show a converged solution while the former one results in acceptale results.

Table 3
L 2 - error between the PINN and the numerical solution of θ(η) for different number of neurons versus different Prandtl numbers when the depth
is fixed at D = 6. The value of inifinity and mesh size are considered as η∞=8.0 and Δ = 0.05, respectively.

increasing the number of layers should be accordingly followed by
increasing the width of the network, or one can increase the number of
layers while the width is fixed.
Table 2 provides insight into this finding, dictating a minimum width
which below that the network cannot precisely estimate the results, e.g.,
(w~80, at, D = 4). It is clearly shown that although 80 neurons are
sufficient for two and four layers structure, however, increasing the
number of layers to six requires adding more neurons (w > 100, e.g., 110
neurons) to satisfy the ODE (Eq. (10)). Finding a relation between the
number of neurons and layers to obtain a reliable model is out of the
focus of this study. However, we should mention that the provided
conclusion is not general and is limited to this problem (Blasius flow). In
Table 2, the highlighted yellow cells show the wrong solutions and (¡)
specifies a range of neurons examined at each layer. The data at each cell
is attributed to the number of neurons written in the parenthesis (bold
number). It is found that w = 150 is sufficient for the networks even by
eight layers (D = 8), and the results show higher accuracy for D = 6
[L 2 -error: (2.13E-4, 1.62E-4 and 1.25E-3) for f(η), f′ (η) and f"(η)
respectively]. Therefore, (150, 6) is selected, and hereafter, we continue
the analysis by this structure. The two conclusions might be drawn from
Table 2, including: 1) at a fixed number of layers, adding the neurons is
likely to resolve the convergence issue, and 2) seems that there is a
correlation between the number of layers and neurons such that adding
more layers requires considering extra neurons at each layer. Fig. 3D
highlights this finding by showing how the loss values are a function of
neurons for D = 6. The dashed lines in Fig. 3A correspond to the
particular sets of neurons and layers (w, D) resulted in the wrong esti­
mations whose loss functions do not fall with respect to increasing the
epochs and are not trained correctly (nearly straight lines in Fig. 3D or
the yellow cells in Table 2). We also examined the role of learning rate
hyperparameter. Fig. 3C demonstrates that the acceptable limit to
obtain accurate results achieved when λ ≤ 10− 4. As it is expected, a
higher value of λ leads to overshooting, such that the network is unable
to find the minimum loss function.
It should be emphasized that the results tabulated in Table 2 are
corresponding to the solution of Eq. (10). Given that Eqs.(10–11) are
coupled ODEs, and only one network is responsible for solving both
these ODEs simultaneously, the required structures to obtain the
acceptable estimated solutions are likely to be different from those
mentioned in Tables 1 and 2. Besides, the Prandtl (Pr) number in Eq.
(11) enhances the nonlinearity, and its effect needs to be examined
separately to solve the coupled Eqs. (11− 12) with their corresponding
boundary conditions.
At first, the structure is set to (150, 6) because of its demonstrated
capability to solve Eq. (10). Table 3 exhibits that this set can accurately
solve the thermal equation at Pr =1. However, the network is not trained
well at higher Prandtl numbers (10 and 100), and the equations are not
satisfied since the Pr number enhances the ODE’s complexity. Interest­
ingly, adding more neurons does not help the network to get trained well
at this range of Prandtl number unless the width is significantly
increased to w = 300. (Note that we did not examine the values between
250 and 300, and the increment value was chosen as 50). Since w = 300
is sufficient to address the problem arising from the high nonlinear terms
induced by increasing the Pr number, (300, 6) is selected as a suitable
structure to solve all the coupled ODEs related to the cases (I) and (II)
investigated in this study. The results obtained in Fig. 3B depict the
temperature and temperature gradient versus η, for different Pr
numbers. The results agree well with those obtained by the numerical
method and clearly show that an increase in the Pr number leads to a
decrease in thermal boundary layer thickness and sharp temperature
gradient at the wall, following the Pr definition, which is the ratio of the
hydrodynamic to thermal boundary layer thickness. Although the loss
function is around ~10− 5, however, the L 2 - errors are comparatively
large at Pr = 10 and 100.
So far, we solved the first case study and tried to examine the
network structure leading to acceptable results. The second problem in
the category of the boundary layer concept is the Falkner-Skan wedge
flow (Eqs. (28)-(29)). The ODEs are similar to the Blasius flow. An
additional term in Eq. (28) represents the effect of wedge angle on the
flow field and its subsequent consequence on thermal distribution (Eq.
(29)). Unlike the previous case, we set the infinity at η∞=6.0 since
indeed it is above the boundary layer thickness, as can be seen in Fig. 4

8
H. Bararnia and M. Esmaeilpour International Communications in Heat and Mass Transfer 132 (2022) 105890

Fig. 4. Results corresponding to the Case (II): Falkner-Skan wedge flow. (A) and (B) are the estimated results obtained by PINN (300, 6) for the flow field. (C) and (D)
are the temperature and temperature gradient distribution versus different Pr numbers and wedge angles.

required to decrease the loss function and close the solution. Fig. 4
Table 4 demonstrates the flow and temperature distribution at different wedge
L 2 - error between the estimated and the numerical solution. The first row angles and Pr numbers. The results show a good match with numerical
shows Falkner-Skan wedge flow related errors, and other rows show errors
data. It is worth pointing out that the above results could also be ach­
attributed to the Natural convection problem. All the values deduced from a (10,
ieved by employing only two layers instead of six layers (in both Cases I
2) network excep the last row (Pr = 5) where (10, 4) was used.
& II), but the minimum requirement in terms of the number of neurons
f f′ f′ ′ θ θ′ needs to be met. The minimum requirement for obtaining a reliable
β = 0.25, Pr = 100 9.76E- 2.52E- 2.07E- 6.58E- 1.61E- solution for the Blasius flow (Pr = 100, β=0.0) and Falkner-Skan at (Pr
04 04 03 02 01 = 100, β=0.25), with only two layers, are (200,10), respectively. At this
Pr = 0.5 4.53E- 5.13E- 1.75E- 9.68E- 4.86E-
high magnitude of Prandtl number, the noticeable effect of angle is
03 03 03 04 04
Pr = 1.0 1.70E- 1.93E- 1.61E- 5.50E- 6.81E- significant, and it is not limited to D = 2 such that the relative reduction
03 03 03 04 04 is also expected for D = 6. A comparison with Table 3 shows that the
Pr = 5.0 1.99E- 2.85E- 1.63E- 1.00E- 2.00E- required number of neurons with six layers for these two cases is
03 03 03 03 03
(300,200).
The last problem is related to the natural convection phenomenon
(Note that one can also take η∞= 8.0). Interestingly, a (200, 6) network (Eqs. (41–42)). Unlike the previous cases, where the flow changes the
could also satisfy the ODEs with the L 2 - errors in the acceptable range, temperature distribution, the flow is induced by a temperature gradient
e.g., at Pr = 100 and β = 0.25, the corresponding errors are (3.03E-02, which results in fully coupled ODEs system. Decreasing the Pr number
3.48E-02, 7.15E-04, 8.11E-04 and 1.25E-03) for θ, θ′ , f, f′ , f", respec­ leads to an increase in thermal boundary layer thickness such that at Pr
tively. The reason behind this considerable discrepancy stems from the = 0.2, an acceptable guess for infinity boundary should be approxi­
effects of the shorter boundary value (η∞=6.0) and the wedge angle (β) mately beyond (η∞ > 10.0). Therefore, at a low Pr number, one must
such that when β approaches zero (horizontal flat plate), ODEs (28–29) carefully adjust the infinity value. Here, we limit the solution to the Pr
become similar to the ODEs (10–11), and therefore more neurons are numbers ranging from 0.5 to 5. Similar to the previous cases, a network

9
H. Bararnia and M. Esmaeilpour International Communications in Heat and Mass Transfer 132 (2022) 105890

Fig. 5. Results corresponding to the Case (III): Natural convection. (A) and (B) are the estimated results obtained by PINN (10, 4) for the flow field. (C) and (D) are
the temperature and temperature gradient distribution versus different Pr numbers.

Fig. 6. Cross-validation tests of the PINN model for Blasius problem. (A) K-fold cross-validation’s results at Pr = 1 and (B) predicted results obtained by hold-out
technique at Pr = 10.

10
H. Bararnia and M. Esmaeilpour International Communications in Heat and Mass Transfer 132 (2022) 105890

Table 5 revealed that, according to the number of hidden layers that construct a
L 2 - errors corresponding to the cross-validation (K-fold) and hold-out methods. network, there was a minimum requirement in terms of the number of
The (K-fold) method is applied to Blasius problem at Pr = 1 and hold-out method neurons at each layer, below which the model was unable to predict the
at Pr = 10. solutions. Also, increasing the Prandtl number magnified the complexity
CV-1 CV-2 CV-3 CV-4 CV-5 Hold-out such that more neurons were required for nonlinear data mapping. The
method possibility of the overfitting issue was also discussed through two cross-
f 1.37E- 3.76E- 6.84E- 2.67E- 7.24E- 1.57E-03 validation techniques, and it was found that the predicted test data by
03 04 04 03 04 PINN matched reasonably well with the numerical results.
θ 8.90E- 4.03E- 6.83E- 2.93E- 1.62E- 1.52E-01
03 03 03 02 03
Credit author statement

Term Author 1*,# Author 2*,#

structure like (10,2) leads to results with acceptable L 2 - error (Table 4)
Conceptualization HB ME
except to Pr = 5 which the number of layers was doubled to increase the Methodology / Study design HB ME
accuracy. Fig. 5 displays the temperature and fluid flow profiles for Software HB ME
different Prandtl numbers and their corresponding numerical solutions. Validation HB ME
Formal analysis HB ME
The results above demonstrate that PINN can effectively solve the
Investigation HB ME
boundary layer problems with infinity boundary conditions. It is also Resources HB ME
shown that in some cases, at a high Pr number or deep network struc­ Data curation HB ME
ture, the necessity of using a wide network seems crucial to address the Writing – original draft HB ME
complexity of the problem (e.g., Blasius flow) considering that utilizing Writing – review and editing HB ME
Visualization HB ME
a large number of neurons in each hidden layer highlights the possibility
Supervision HB ME
of overfitting. Although satisfying the underlying governing equations Project administration
as one of the main constraints in the definition of the total loss function Funding acquisition
can act as a regularization mechanism [46], we study the overfitting *
Use tick mark (√) to indicate contribution, a cross (X) to indicate no
issue at a specifically wide network structure (w = 300). In this respect,
contribution and NA where not applicable.
in contrary to the previous cases, the whole dataset is not used for #
Use author initials to declare the contributions to the manuscript.
training purposes. Instead, two cross-validation methods were used: 1)
hold-out, and 2) K-fold split methods, to evaluate the accuracy of the Declaration of Competing Interest
models. It should be mentioned that neither the drop-out option nor a
regularization parameter is used in our modeling. We select the first case The authors declare that they have no known competing financial
(Blasius problem at Pr=1 and Pr=10) since a deep network (300, 6) is interests or personal relationships that could have appeared to influence
employed to obtain the results shown in Fig. 3. The first method requires the work reported in this paper.
splitting the data set (η[0, 8], fNum, θNum) into two sections: training
(60%) and testing (40%) portions. The PINN model is built by using the References
training portion and subsequently evaluated by the test section. In the
second method, the whole data is split into five subsets (5-fold), and the [1] B.M. Abbagoni, H. Yeung, Non-invasive classification of gas–liquid two-phase
PINN model is built by using one of these folds as a test section and the horizontal flow regimes using an ultrasonic Doppler sensor and a neural network,
Meas. Sci. Technol. 27 (8) (2016), 084002, https://doi.org/10.1088/0957-0233/
rest as a training portion. The procedure is repeated five times to ensure 27/8/084002.
that each data is included once in the test set. We use the shuffle option [2] A. Agrawal, A. Choudhary, Deep materials informatics: applications of deep
with random_state = 1.0 in the Sklearn package of Python. As shown in learning in materials science, MRS Commun. 9 (3) (2019) 779–792, https://doi.
org/10.1557/mrc.2019.73.
Fig. 6A, the predictions of test data for each CV1-5 collapse with corre­
[3] F. Ahmad, W.H. Al-Barakati, An approximate analytic solution of the Blasius
sponding numerical results. Also, according to the Table 5, the mean problem, Commun. Nonlinear Sci. Numer. Simul. 14 (4) (2009) 1021–1024,
value of L 2 - errors attributed to the f and θ at Pr = 1 are 1.16E-03 and https://doi.org/10.1016/j.cnsns.2007.12.010.
1.01E-02, respectively, signifying the model’s accuracy in predicting the [4] I. Ahmad, M. Bilal, Numerical solution of Blasius equation through neural networks
algorithm, Am. J. Comput. Math. 4 (3) (2014) 223–232, https://doi.org/10.4236/
temperature distribution compared to the corresponding value in ajcm.2014.43019.
Table 3. Fig. 6B also exhibits the capability of the model performed by a [5] E. Alizadeh, M. Farhadi, K. Sedighi, H.R. Ebrahimi-Kebria, A. Ghafourian, Solution
hold-out method in predicting the unseen data (test data), highlighting of the Falkner Skan equation for wedge by Adomian decomposition method,
Commun. Nonlinear Sci. Numer. Simul. 14 (3) (2009) 724–733, https://doi.org/
the low chance of overfitting in this case. 10.1016/j.cnsns.2007.11.002.
[6] B. Arnold, T. Lutz, E. Krämer, Design of a boundary-layer suction system for
5. Conclusion turbulent trailing-edge noise reduction of wind turbines, Renew. Energy 123
(2018) 249–262, https://doi.org/10.1016/j.renene.2018.02.050.
[7] H. Blasius, Grenzschichten in Flüssigkeiten mit kleiner Reibung, Druck von BG
This study implemented the physics-informed neural networks to Teubner, 1908.
solve three benchmark boundary layer flow and thermal problems. The [8] S.L. Brunton, B.R. Noack, P. Koumoutsakos, Machine learning for fluid mechanics,
Annu. Rev. Fluid Mech. 52 (2020) 477–508, https://doi.org/10.1146/annurev-
solutions were based on training the networks with the loss functions fluid-010719-060214.
responsible for satisfying both boundary conditions and the governing [9] H. Chen, O. Engkvist, Y. Wang, M. Olivecrona, T. Blaschke, The rise of deep
equations. The essential features of the technique, such as learning rate, learning in drug discovery, Drug Discov. Today 23 (6) (2018) 1241–1250, https://
doi.org/10.1016/j.drudis.2018.01.039.
depth, and width of the structure were discussed in details. It was found
[10] Y. Chen, L. Lu, G.E. Karniadakis, L. Dal Negro, Physics-informed neural networks
that the networks with a learning rate below 10− 4 were able to predict for inverse problems in nano-optics and metamaterials, Opt. Express 28 (8) (2020)
the solutions accurately. The estimated solutions of the trained PINN 11618–11633, https://doi.org/10.1364/OE.384875.
[11] N. Cohen, O. Sharir, A. Shashua, On the expressive power of deep learning: A
were compared with numerical results. It was shown that the predictions
tensor analysis, in: 29th Annual Conference on Learning Theory, PMLR, 2016,
were in good agreement with those obtained by the numerical method. pp. 698–728.
Moreover, it was demonstrated that an assumed value for infinity [12] B. Colvert, M. Alsalman, E. Kanso, Classifying vortex wakes using neural networks,
boundary conditions should be approximately greater than the bound­ Bioins. Biomimet. 13 (2) (2018), 025003, https://doi.org/10.1088/1748-3190/
aaa787.
ary layer thickness. However, considering a higher value as an alter­ [13] M. Dissanayake, N. Phan-Thien, Neural-network-based approximations for solving
native value for infinity required more neurons at each layer. The results partial differential equations, Commun. Numer. Methods Eng. 10 (3) (1994)
195–201, https://doi.org/10.1002/CNM.1640100303.

11
H. Bararnia and M. Esmaeilpour
[14] Z. Du, M.S. Selig, The effect of rotation on the boundary layer of a wind turbine
blade, Renew. Energy 20 (2) (2000) 167–181, https://doi.org/10.1016/S0960-
1481(99)00109-3.
[15] R. Eldan, O. Shamir, The power of depth for feedforward neural networks, in: 29th
Annual Conference on Learning Theory, PMLR, 2016, pp. 907–940.
[16] M. Esmaeilpour, D.D. Ganji, Application of He’s homotopy perturbation method to
boundary layer flow and convection heat transfer over a flat plate, Phys. Lett. A
372 (1) (2007) 33–38, https://doi.org/10.1016/j.physleta.2007.07.002.
[17] V.M. Falkner, S.W. Skan, Some approximate solutions of the boundary layer
equations, Philos. Mag. 12 (1931) 865–896, https://doi.org/10.1080/
14786443109461870.
[18] B.G. Goh, N.O. Hodas, A. Vishnu, Deep learning for computational chemistry,
J. Comput. Chem. 38 (16) (2017) 1291–1307, https://doi.org/10.1002/jcc.24764.
[19] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT press, 2016.
[20] I. Grant, X. Pan, The use of neural techniques in PIV and PTV, Meas. Sci. Technol. 8
(12) (1997) 1399–1405, https://doi.org/10.1088/0957-0233/8/12/004.
[21] K. Hornik, M. Stinchcombe, H. White, Multilayer feedforward networks are
universal approximators, Neural Netw. 2 (5) (1989) 359–366, https://doi.org/
10.1016/0893-6080(89)90020-8.
[22] L. Howarth, On the solution of the laminar boundary layer equations, Proc. R. Soc.
Lond. Ser. A 164 (919) (1938) 547–579, https://doi.org/10.1098/rspa.1938.0037.
[23] D. Izzo, M. Märtens, B. Pan, A survey on artificial intelligence trends in spacecraft
guidance dynamics and control, Astrodynamics 3 (4) (2019) 287–299, https://doi.
org/10.1007/s42064-018-0053-6.
[24] W.M. Kays, M.E. Crawford, Convective Heat and Mass Transfer, 3rd edition,
McGraw–Hill, New York, 1993.
[25] M.F. Kelly, P.A. Parker, R.N. Scott, The application of neural networks to
myoelectric signal analysis: a preliminary study, IEEE Trans. Biomed. Eng. 37 (3)
(1990) 221–230, https://doi.org/10.1109/10.52324.
[26] H.K. Kuiken, Free convection at low Prandtl number, J. Fluid Mech. 37 (4) (1969)
785–798, https://doi.org/10.1017/S0022112069000887.
[27] I.E. Lagaris, A. Likas, D.I. Fotiadis, Artificial neural networks for solving ordinary
and partial differential equations, IEEE Trans. Neural Netw. 9 (5) (1998)
987–1000, https://doi.org/10.1109/72.712178.
[28] Y. Lian, W. Shyy, Laminar-turbulent transition of a low Reynolds number rigid or
flexible airfoil, AIAA J. 45 (7) (2007) 1501–1513, https://doi.org/10.2514/
1.25812.
[30] S.J. Liao, An approximate solution technique not depending on small parameters: a
special example, Int. J. Non-Lin. Mech. 30 (3) (1995) 371–380, https://doi.org/
10.1016/0020-7462(94)00054-E.
[31] S. Liao, A. Campo, Analytic solutions of the temperature distribution in Blasius
viscous flow problems, J. Fluid Mech. 453 (2002) 411–425, https://doi.org/
10.1017/S0022112001007169.
[32] J. Ling, A. Kurzawski, J. Templeton, Reynolds averaged turbulence modelling
using deep neural networks with embedded invariance, J. Fluid Mech. 807 (2016)
155–166, https://doi.org/10.1017/jfm.2016.615.
[33] S.X. Lv, L. Peng, L. Wang, Stacked autoencoder with echo-state regression for
tourism demand forecasting using search query data, Appl. Soft Comput. 73 (2018)
119–133, https://doi.org/10.1016/j.asoc.2018.08.024.
[34] M. Milano, P. Koumoutsakos, Neural network modeling for near wall turbulent
flow, J. Comput. Phys. 182 (1) (2002) 1–26, https://doi.org/10.1006/
jcph.2002.7146.
[35] W. Munters, J. Meyers, An optimal control framework for dynamic induction
control of wind farms and their interaction with the atmospheric boundary layer,
Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 375 (2091) (2017) 20160100,
https://doi.org/10.1098/rsta.2016.0100.
[36] H. Mutuk, A neural network study of Blasius equation, Neural. Process. Lett. 51
(2020) 2179–2194, https://doi.org/10.1007/s11063-019-10184-9.
12
International Communications in Heat and Mass Transfer 132 (2022) 105890
[37] A. Oishi, G. Yagawa, Computational mechanics enhanced by deep learning,
Comput. Methods Appl. Mech. Eng. 327 (2017) 327–351, https://doi.org/
10.1016/j.cma.2017.08.040.
[38] F. Olaiya, A.B. Adeyemo, Application of data mining techniques in weather
prediction and climate change studies, Int. J. Inform. Eng. Electr. Bus. 4 (1) (2012)
51–59, https://doi.org/10.5815/ijieeb.2012.01.07.
[39] S. Ostrach, An analysis of laminar free-convection flow and heat transfer about a
flate plate parallel to the direction of the generating body force, NACA Rep. 1111
(1953).
[40] G. Pandey, A. Dukkipati, To go deep or wide in learning?, in: Proceedings of the
Seventeenth International Conference on Artificial Intelligence and Statistics,
PMLR vol. 33, 2014, pp. 724–732.
[41] L. Peng, S. Liu, R. Liu, L. Wang, Effective long short-term memory with differential
evolutionalgorithm for electricity price prediction, Energy 162 (2018) 1301–1314,
https://doi.org/10.1016/j.energy.2018.05.052.
[42] H.A. Pierson, M.S. Gashler, Deep learning in robotics: a review of recent research,
Adv. Robot. 31 (16) (2017) 821–835, https://doi.org/10.1080/
01691864.2017.1365009.
[43] L. Prandtl, Über Flussigkeitsbewegung bei sehr kleiner Reibung. Verhandl. III,
Internat. Math.-Kong., Heidelberg, Teubner, Leipzig vol. 2, 1904, pp. 484–491.
[44] S. Raayai-Ardakani, G.H. McKinley, Drag reduction using wrinkled surfaces in high
Reynolds number laminar boundary layer flows, Phys. Fluids 29 (9) (2017),
093605. http://hdl.handle.net/1721.1/119866.
[45] M.T. Rad, A. Viardin, G.J. Schmitz, M. Apel, Theory-training deep neural networks
for an alloy solidification benchmark problem, Comput. Mater. Sci. 180 (2020),
109687, https://doi.org/10.1016/j.commatsci.2020.109687.
[46] M. Raissi, P. Perdikaris, G.E. Karniadakis, Physics informed deep learning (part I):
data-driven solutions of nonlinear partial differential equations, arXiv Prepr.
(2017) arXiv:1711.10561.
[47] M. Raissi, P. Perdikaris, G.E. Karniadakis, Physics-informed neural networks: a
deep learning framework for solving forward and inverse problems involving
nonlinear partial differential equations, J. Comput. Phys. 378 (2019) 686–707,
https://doi.org/10.1016/j.jcp.2018.10.045.
[48] M. Raissi, A. Yazdani, G.E. Karniadakis, Hidden fluid mechanics: learning velocity
and pressure fields from flow visualizations, Science 367 (6481) (2020)
1026–1030, https://doi.org/10.1126/science.aaw4741.
[49] F. Sarghini, G. De Felice, S. Santini, Neural networks based subgrid scale modeling
in large eddy simulations, Comput. Fluids 32 (1) (2003) 97–108, https://doi.org/
10.1016/S0045-7930(01)00098-6.
[50] E. Schmidt, W. Beckmann, Das Temperatur-und Geschwindigkeitsfeld vor einer
Wärme abgebenden senkrechten Platte bei natürlicher Konvektion, Tech. Mech.
Thermodyn. 1 (11) (1930) 391–406, https://doi.org/10.1007/BF02660553.
[51] P.A. Srinivasan, L. Guastoni, Schlatter Azizpour, P., Vinuesa, R., Predictions of
turbulent shear flows using deep neural networks, Phys. Rev. Fluids 4 (5) (2019),
054603, https://doi.org/10.1103/PhysRevFluids.4.054603.
[52] C.L. Teo, K.B. Lim, G.S. Hong, M.H.T. Yeo, A neural net approach in analyzing
photograph in PIV, in: Proceeding IEEE International Conference on Systems, Man
and Cybernetics, Charlottesville, VA, USA vol. 3, 1991, pp. 1535–1538, https://
doi.org/10.1109/ICSMC.1991.169906.
[53] T. Wang, Z. Huang, Z. Sun, G. Xi, Reconstruction of natural convection within an
enclosure using deep neural network, Int. J. Heat Mass Transf. 164 (2021),
120626, https://doi.org/10.1016/j.ijheatmasstransfer.2020.120626.
[54] T. Xie, S.M. Ghiaasiaan, S. Karrila, Artificial neural network approach for flow
regime classification in gas–liquid–fiber flows based on frequency domain analysis
of pressure signals, Chem. Eng. Sci. 59 (11) (2004) 2241–2251, https://doi.org/
10.1016/j.ces.2004.02.017.
[56] N. Zobeiry, K.D. Humfeld, A physics-informed machine learning approach for
solving heat transfer equation in advanced manufacturing and engineering
applications, Eng. Appl. Artif. Intell. 101 (2021), 104232, https://doi.org/
10.1016/j.engappai.2021.104232.