Nuclear Engineering and Design 326 (2018) 220–233

Contents lists available at ScienceDirect

Nuclear Engineering and Design

journal homepage: www.elsevier.com/locate/nucengdes

Numerical error analysis for three-dimensional CFD simulations in the two- T

room model containment THAI+: Grid convergence index, wall treatment
error and scalability tests
⁎
A. Mansour , E. Laurien
University of Stuttgart, Institute of Nuclear Technology and Energy Systems, Pfaffenwaldring 31, 70569 Stuttgart, Germany

G RA P H I C A L AB S T R A C T

A R T I C L E I N F O A B S T R A C T

Keywords: This paper presents a step to define Best Practice Guidelines for CFD simulations in nuclear containment ap-
Grid Convergence Index GCI plications and other disciplines with similar complex geometries. For the two-room model containment THAI+, a
Spatial discretization error three-dimensional natural convection flow simulation was performed using the CFD package Ansys CFX 16.1.
Natural convection For the quantification of the numerical error, the applicability of three versions of the Grid Convergence Index
Model containment THAI+
GCI to the complex flow field was tested. Those are the Standard method REM, the Blend Factor Method BFM
Wall treatment
and the Least Squares Method LSQ. Using these methods, the spatial discretization errors were quantified on six
Scalalbility
grids with different refinement levels up to 39.731·106 elements. Besides, the model error due to the different
wall treatment approaches was also quantified. For this, the simulation results on grids with different y+ ranges
were compared. Relative errors of approximately 58.40%, 7.2%, 8.43% and 0.96% were detected in the volume-
integrated vorticity, temperature, mass flows and pressure between the simulations using the low-Reynolds
approach (y+ < 1) and the wall function of the SST model (y+ > 30). The parallel performance of the cal-
culations was also investigated on a CRAY XC-40 using four grids with different resolutions. The maximum
speedup was achieved on approximately 1838 computational cores on the finest grid with 83·106 elements and
24·106 nodes and on 562 cores on the coarsest grid with 1.268·106 elements and 0.347·106 nodes. The insight and
results of the GCI, wall treatment and scalability studies can be used as guidelines, on which future CFD con-
tainment simulations can be based. Finally, the comparison of the simulation results with the experimental data
was carried out.

⁎
Corresponding author.
E-mail address: abdennaceur.mansour@ike.uni-stuttgart.de (A. Mansour).

https://doi.org/10.1016/j.nucengdes.2017.11.010
Received 20 February 2017; Received in revised form 16 October 2017; Accepted 3 November 2017
Available online 20 November 2017
0029-5493/ © 2017 Elsevier B.V. All rights reserved.
A. Mansour, E. Laurien Nuclear Engineering and Design 326 (2018) 220–233

Nomenclature u Velocity Vector [m/s]

Uf LSQ Uncertainty [−]
Symbols v Vertical Velocity Component [m/s]
Vorave Volume-averaged Vorticity [1/s]
α Error Constant [−] wi Wight of the grid spacing hi [−]
β Blend Factor [−] y+ Dimensionless Wall Distance [−]
c Mass Fraction [−]
E1 Estimator of the Richardson Extrapolation [−] Abbreviations
ε CFD Error Estimator [−]
φ Relative Humidity [−] BFM Blend Factor Method
Fs Safety Factor [−] CFD Computational Fluid Dynamics
g Earth Gravity [m/s2] CPU Central Processing Unit
h Grid Spacing [m] GCI Grid Convergence Index
ṁ tube Pipe Mass flows [kg/s] HLRS High Performance Computing Center Stuttgart
p Order of Accuracy [−] HTC Heat Transfer Coefficient
p Pressure [bar] LSQ Least Squares Method
qjRe Turbulent Heat Flux [W/m2] MP Measurement Point
ρ Density [kg/ m3] OC Oscillatory Convergence
r Refinement Ratio [−] PAD Parallel Attachable Drum
s Signum Function [−] PWR Pressurized Water Reactor
Sp Speedup [−] REM Richardson Extrapolation Method
τijRe Reynolds Stress Tensor [N/(m2)] THAI Thermal-hydraulics, H2, Iodine and Aerosols
t Time [s] URANS Unsteady Reynolds Averaged Navier-Stokes equations
T Temperature [°C]

1. Introduction geometry complexity. For this reason, an efficient parallel computing is

Besides many other disciplines, the methods of computational fluid
dynamics CFD have been recently used in nuclear applications in order
to understand and predict the flow processes in model containments
(Babić et al., 2005; Zirkel et al., 2008; Houkema et al., 2008). Those
methods are expected to have a good accuracy and a wide range of
applicability and can present an alternative to the traditional Lumped
Parameter LP Codes used in the reactor safety analysis (IAEA, 2003),
since these LP methods may be characterized by a strong user effect
which could have an influence on the calculation accuracy (NEA,
2007). Nevertheless, CFD calculations require much higher computing
time than the LP codes due to the high complexity of the flow processes
and numerical grids in containment simulations. For this reason, dif-
ferent participants at the ISP-47 used relatively coarse numerical grids
for their simulations and no investigations of the spatial discretization
error were done (NEA, 2007).
For the quantification of this error, the Grid Convergence Index GCI
has been introduced by Roache (1998). This method has been re-
commended for the estimation of the discretization error (Celik et al.,
2008) and has been used in many disciplines, such for example in
aerospace applications (Carozza, 2016), in biomedical problems
(Longest and Vinchurkar, 2007) and in turbomachinery applications
(Lenarcic et al., 2015). Besides CFD, the GCI method has been also
applied in other numerical branches, such as the solid and structural
mechanics field (Kwaśniewski, 2013). The Grid Convergence Index was
also recommended by the best practice guidelines for CFD in nuclear
reactor safety applications (NEA, 2007). In a previous work (Mansour
and Laurien, 2016), the GCI approach as described by Celik et al.
(2008) was applied for a natural convection flow simulation in the two-
room model containment THAI+ to investigate the spatial discretiza-
tion error. This resulted in a case of a noisy grid convergence, where
very high values of the observed accuracy order were measured. To deal
with this issue, the Blend Factor Method BFM which is based on the
High Resolution Scheme was introduced in order to determine the
averaged accuracy order of the code. Another approach to deal with
this issue is the Least Squares Method (Eça and Hoekstra, 2014).
However, complex and large grids have to be used in order to carry
out a GCI study for containment simulations due to the physics and
very meaningful in order to achieve an optimum use of the computa-
tional resources. Scalability tests for Ansys CFX 14.0 were performed on
the Cray XE6 Hermit Cluster at the High Performance Computing
Center HLRS in Stuttgart (Zhang and Laurien, 2015). For this, a grid
with approx. 10.2·106 elements of a Pressurized Water Reactor con-
tainment was used. The parallel performance of the calculations was
observable away from the ideal performance. For 80 cores, the ratio of
the maximum speedup to the ideal one was equal to 35/80. However,
the new versions of Ansys CFX, including the version 16.1 used in this
work, have been improved in terms of parallel performance (Kainz,
2016).
The main aim of this paper is to define Best practice Guidelines for
CFD containment simulations. This is carried out by the estimation of
the numerical discretization error in a natural convection flow simu-
lation based on the theory of the Grid Convergence Index. For this,
three main versions of the GCI, which are the Standard method REM,
the Blend Factor method BFM and the Least Squares method LSQ will
be compared and the spatial discretization errors for six grids with
different numerical resolutions will be estimated. In addition, the model
error resulting from the use of the automatic wall function will be
quantified and the comparison of the simulation results with the ex-
perimental data will be carried out. Furthermore, tests of the scalability
on parallel computers will be performed using four grids with different
refinement levels in order to define an optimal use of the computational
resources for each of these levels.
2. Computational model
2.1. Governing equations
Humid air is a mixture of two components which are steam or water
vapor and air. These two components have the same mean properties:
velocity, temperature and pressure. The transport equations for this
mixture are described by the Unsteady Reynolds
Averaged Navier-Stokes equations URANS. These are based on
Favre Averaging, whereby a quantity u is decomposed into a mean
value u and a fluctuation term u′.

221
A. Mansour, E. Laurien Nuclear Engineering and Design 326 (2018) 220–233

∼)
∂ρ ∂t ∂ (ρ u where ΔVj is the volume of each grid element.
i
=0 ;i,j = 1,2,3 h h
+ ∂x i (1) After calculating the grid refinement ratios r21 = h2 and r32 = h3 , the
1 2
observed accuracy order p will be determined. Finally, the GCI values
∼ ∼
∂u
⎡ ∂ (ρ u
∼ ) + ∂ (ρ u
∼u ∼ ⎤ ∂p ∂ ⎡ ⎛ ∂u j⎞ Re⎤ can be calculated using Eq. (8). The three methods described below
i j i )⎥ = − + ⎢μ⎜ i + ⎟ + τij
⎥ + ρ gi
⎢ ∂t ∂x j ∂x i ∂x j ∂x j ∂x i ⎠ were applied to calculate the GCI values or the discretization error
⎣ ⎦ ⎣ ⎝ ⎦
(2) depending on the value of the accuracy order p.

∼h ∼
⎡ ∂ ∼ ∂p ∂ (ρ u j )⎤ ∂ 2.2.1. Richardson-extrapolation-based accuracy order REM
⎢ ∂t (ρ h )− ∂t + = [q + qjRe]
∂x j ⎥ ∂x j j (3)
In this method, the accuracy order p is calculated using the fol-
⎣ ⎦
lowing expression (Celik et al., 2008)
u , p and h are the ensemble-averaged values of the velocity, pressure
f3 − f2 rp −s
and enthalpy. gi stands for the gravity component and the indexes i,j ln f2 − f1
+ ln ⎛ 21p ⎞
∼ ⎝ r32 − s ⎠
represent the three position vectors. τijRe = −ρ ui″ u″j in Eq. (2) stand for p=
∼ ln(r21 ) (10)
the components of the Reynolds stress tensor while qjRe = ρ cuj T ′ in Eq.
(3) are the turbulent heat fluxes. τijRe and qjRe are modeled by the SST with
turbulence model used in this simulation. ρ is the ensemble-averaged f −f
density of the mixture while the density ρcomp of each component is s = sign ⎛⎜ 3 2 ⎟⎞.
defined as follows: ⎝ f2 −f1 ⎠ (11)

pref Eq. (10) can be solved using a fixed-point iteration algorithm. It has
ρcomp = been derived from the Richardson Extrapolation theory (Celik et al.,
R comp T (4)
2008). The calculated value of p will be used to determine the GCI
R comp is the gas constant of the components and pref = 2bars is the re- values according to Eq. (8).
ference pressure. The initial mass fractions of steam and air and are
c vap = 0.289 and cair = 0.711. 2.2.2. Blend factor method BFM
In previous works, the use of the REM method resulted in very high
2.2. Grid convergence index values of the accuracy order p. This led to very low and unrealistic error
estimates (Mansour and Laurien, 2016). For this reason, we developed
The method of the Grid Convergence Index was first proposed by the Blend Factor Method BFM in order to provide more reasonable
Roache (1998). This method is based on the Richardson Extrapolation, accuracy order values which are not necessarily equal to the theoretical
in which an estimation of the exact solution can be provided order of the code (mostly assumed to be equal to 2).
In Ansys CFX, one can set the order of the advection and transient
f1 −f2
fexact ≅ f1 + . schemes as well as the turbulence numerics, i.e. the equations used in
r p−1 (5)
the turbulence models, to first or second order. It is also possible to use
here, f1 and f2 are numerical solutions of the investigated quantities on a hybrid scheme, which blends between zero and one depending on the
two different grids with discrete spacings h1 (fine grid) and h2 (coarse complexity of the numerics. In the current calculations, the high re-
h
grid). r = h2 is the grid refinement ratio and p stands for the accuracy solution advection scheme is used for the spatial discretization (Barth
1
order of the discretization scheme. The objective of the Richardson and Jespersen, 1989). This is an adaptive method which uses a blend
extrapolation consists, as stated above, in providing an approximation factor β. This blend factor varies in the range between 0 and 1 and aims
fexact of the exact solution using the numerical solutions f1 and f2 . to maintain the accuracy order as close to second order as possible. In
The relative numerical error between fexact and the fine grid solution numerically stable flow regions, β is set to 1. In this case, the second
f1 is described by order central differencing scheme is used. However, in regions char-
fexact −f1 acterized by numerical instabilities, β will be set to 0 and the first order
ε
E1 = = , upwind scheme will be used to ensure more robustness. The high re-
f1 r p−1 (6)
solution scheme is also used in the turbulence numerics. For the dis-
where ε is the relative error between f1 and f2 , which is defined as cretization of the transient terms, the second order backward euler
method is employed.
f1 −f2
ε= . In CFX, there are a total of six blend factors for multicomponent
f1 (7) simulations. One for the energy and species, one for each of the mo-
The Grid Convergence Index GCI value corresponds to the numer- mentum/ velocity equations and two blend factors for the turbulence
ical uncertainty, in which the numerical error E1 is multiplied by a eddy frequency and kinetic energy (Günther, 2016), see Table 1.
safety factor Fs : The Blend Factor Method BFM consists in calculating the accuracy
order depending on the values of the blend factor β. As stated above, if
|ε| β is equal to 1 then the accuracy order p will be equal to 2. However, for
GCI = Fs .
r p−1 (8) all other values of β (i.e. β < 1), p will be equal to 1 and the blend
The safety factor Fs is set to 1.25 when more than 2 grids are used
(Roache, 1998). Table 1
Blend Factors in Ansys CFX.
In the current GCI study, the procedure recommended by Celik et al.
(2008) is adopted. Suppose that three grids i = 1, 2, 3 have been be Variable: Blend Factor Equation
generated for our CFD calculation. N1, N2 and N3 are the total element
numbers for grid 1 (fine), grid 2 (middle) and grid 3 (coarse). The first Energy and mass Fractions·Beta Energy and species equations
Velocity u·Beta Momentum equation in x direction
step is to calculate the average spacing for each grid
Velocity v·Beta Momentum equation in y direction
Ni
1/3 Velocity w·Beta Momentum equation in z direction
⎡1 ⎤ Turbulence Eddy Frequency·Beta Equation for the turbulence eddy frequency
hi = ⎢
N
∑ ΔVj ⎥ ,
Turbulence Kinetic Energy·Beta Equation for the turbulence kinetic energy
⎣ i j=1 ⎦ (9)

222
A. Mansour, E. Laurien Nuclear Engineering and Design 326 (2018) 220–233

factor value will merely influence the error constant (Pereira et al., • If 0.5 ⩽ p ⩽ 2
2013).
Using this approach, the accuracy order will be determined in each For this acceptable range of the accuracy order, the discretization
grid element/node i error is determined from δRE in Eq. (14). The values of α and fexact
should be taken, as mentioned before, from the fit with the smallest
2, if β = 1
pi (β ) = ⎧ . standard deviation σ .
⎨
⎩1, otherwise (12)

Using pi (β ) , the mean value of the volume-weighted average accuracy • If p > 2

orders will be then calculated
This large value of p will cause a low unrealistic error estimate, the
n Ni so-called “noisy grid convergence”. In this case, the discretization error
1⎡ 1 ⎞⎤
pave = ∑⎛
n ⎢ j ⎜V
∑ pi (β )⎟ ⎥, will not be determined from Eq. (14), but using the following equations
⎣ ⎝ i=1 ⎠⎦ (13) for first and second order accuracy:
+ δ1 = fi −fexact = αhi i. e . p = 1,
where n stands for the number of grids, V for the volume of THAI and (18)
Ni for the element number in each grid. Eq. (13) will be then used to
assess the GCI values defined in Eq. (8). The BFM method can be also δ2 = fi −fexact = αhi2 i. e . p = 2. (19)
used to determine the observed accuracy order if the first order upwind The values of fexact and α will be estimated from the best of the
scheme or second order central differencing scheme are used. following four fits, which has the smallest standard deviation σ :
n
2.2.3. Least squares method ⎡ ⎤
⎢S1 (fexact ,α ) = ∑ (fi −(fexact + αhi ))2 ⎥ → Minimum,
This method was introduced by Eça and Hoekstra (2014) in order to ⎣ i=1 ⎦ (20)
deal with the so-called ”noisy grid convergence”, where very high or
n
too small values of the observed order of accuracy are obtained. Some ⎡ ⎤
⎢S2 (fexact ,α ) = ∑ (fi −(fexact + αhi2 ))2 ⎥ → Minimum,
contributors to this behavior could be the complexity of the geometry
⎣ i=1 ⎦ (21)
and the complicated high-turbulent flows. In addition, complex un-
structured grids could be another reason for this noisy grid convergence n
⎡ w ⎤
(Eça and Hoekstra, 2014). ⎢S1 (fexact ,α,) = ∑ wi (fi −(fexact + αhi ))2 ⎥ → Minimum,
The method is also based on the error estimation of the Richardson ⎣ i=1 ⎦ (22)
Extrapolation
n
⎡ w ⎤
δRE = fi −fexact = αhip, (14) ⎢S2 (fexact ,α,) = ∑ wi (fi −(fexact + αhi2 ))2 ⎥ → Minimum.
⎣ i=1 ⎦ (23)
where fi is the discrete solution on a mesh with a grid spacing hi , fexact is

• If p < . 5
the estimation of the exact solution from Eq. (5), α is an error constant
and p stands for the observed accuracy order.
To determine the three unknowns in Eq. (14), which are fexact , α and
These small p values will result in large too conservative error es-
p, the Least Squares Method is used. For this, the sum SRE of the de-
timates and that is why the discretization error will be estimated using,
viations between the discrete solutions fi and the model function
in addition to δ1 in Eq. (18) and δ2 in Eq. (19), also the following
(fexact + αhip) is minimized
equation:
n
⎡ ⎤
(fi −(fexact + αhip ))2 ⎥ → Minimum. δ12 = fi −fexact = α1 hi + α2 hi2. (24)
⎢SRE (fexact ,α,p) = ∑
⎣ i=1 ⎦ (15) To calculate fexact and α , two other fits will be used besides the four
here, n stands for the grid number, which should be at least equal to 4 in fits (20)–(23):
order to ensure more accurate error estimation. Eq. (15) can be also n
⎡ ⎤
weighted in the sense, that the data on the fine grids are more accurate ⎢S12 (fexact ,α1,α2) = ∑ (fi −(fexact + α1 hi + α2 hi2 ))2 ⎥ → Minimum,
and more valuable than on the coarse grids. This leads to the weighted ⎣ i=1 ⎦
least squares version (25)
n n
⎡ w ⎤ ⎡ w ⎤
⎢SRE (fexact ,α,p) = ∑ wi (fi −(fexact + αhip ))2 ⎥ → Minimum. ⎢S12 (fexact ,α1,α2) = ∑ wi (fi −(fexact + α1 hi + α2 hi2 ))2 ⎥ → Minimum.
⎣ i=1 ⎦ (16) ⎣ i=1 ⎦
(26)
The weights wi are calculated on the basis of the grid spacing hi
After calculating the discretization error, the numerical uncertainty
1
hi Uf can be now determined. This uncertainty corresponds to the GCI
wi = n 1
, value defined in Eq. (8) in the REM and BFM versions of the grid
∑i = 1 h (17)
i convergence index. To do this, a data range parameter Δf is defined in
The accuracy order is determined by solving Eqs. (15) and (16) and order to evaluate the quality of the data fits:
comparing both solutions. If both equations provide positive solutions fi,max −fi,min
p > 0 , then one should choose the solution of the fit with the smallest Δf = .
n−1 (27)
standard deviation σ (Eça and Hoekstra, 2014). In the case that one of
the solutions is negative, the corresponding fit or model function will The uncertainty is then calculated depending on Δf :
not be considered.
In a second step, the discretization error δ can be calculated using • If σ < Δ f

the following procedure depending on the values of the accuracy order

p:

223
A. Mansour, E. Laurien
1 air and 0.289 for steam. Except the sump, all THAI walls are heated up
Uf = (Fs |δ| + σ ).
fexact (28)
In Eq. (28), δ stands for the discretization error and σ for the standard
deviation. Fs is a safety factor, which should be set to 1.25 if 0.5 ⩽ p ⩽ 2
(i.e. the estimation is reliable) and to 3 in all other cases.
• If σ ⩾ Δ
f
1 ⎛ σ
Uf = ⎜3 (|δ| + σ ) ⎟⎞.
fexact ⎝ Δf ⎠ (29)
3. Numerical method
3.1. THAI+ test facility: Geometry and mesh
THAI+ is a newly constructed two-room test facility which consists
of two vessels: THAI and PAD. THAI (Thermal-hydraulics, H2, Iodine
and Aerosols) has a height of 9.2 m high and a diameter of 3.2 m while
PAD (Parallel Attachable Drum) is 9.8 m high with a diameter of 1.6 m.
In order to enable the flow circulation from one vessel to another, THAI
and PAD are interconnected by DN 500 pipes with a diameter of 0.5 m,
see Fig. 1.
The Ansys DesignModeler was used to create the geometry of
THAI+ based on a CAD model of Becker Technologies GmbH (Fig. 1).
THAI vessel contains several internal installations. In addition to the
open inner cylinder which has a height of 4 m and a diameter of 1.4 m,
there are also condensate trays and gutters. The four trays are placed
between the inner cylinder and the THAI walls and are separated by
open sectors of 30° which ensure flow circulation between the upper
and lower annulus of THAI vessel. The condensate gutters are located in
the upper annulus of THAI and at the bottom the inner cylinder (Freitag
et al., 2016). Due to the geometrical complexity of THAI, i.e the several
internal installations, this vessel represents the plant room of a Pres-
surized Water Reactor while the PAD vessel corresponds to the oper-
ating room.
Using the Ansys Meshing Tool, a 3D grid was generated for THAI+.
This grid consists of tetrahedral elements in the flow volume and prism
layers on the walls see Fig. 2.
3.2. Setup and boundary conditions
The initial temperature and pressure are equal to 92.89 °C and
2 bars. The humid air was saturated at t = 0 s, i.e. the initial relative
humidity amounts to 100%. This results in a mass fraction of 0.711 for
224
Nuclear Engineering and Design 326 (2018) 220–233
to 105 °C (see Fig. 1). A natural convection between the two vessels is
thereby initiated due to the density difference between the hot and cold
humid air in THAI and PAD. Despite the insulation, there were im-
portant heat losses during the experiment. Those were calculated using
heat transfer coefficients, which were estimated using the thermal
conductivity of the different materials and based on empirical values
taken from previous experiments in THAI vessel (Freitag, 2016).
The transient simulations were carried out using the CFD package
Ansys CFX 16.1. The SST turbulence and full buoyancy models were
employed for turbulence and buoyancy. The high resolution advection
scheme is applied to the transport and turbulence equations and a
second order backward euler scheme to the transient terms.
4. Results
4.1. Grid convergence Index
The GCI study was performed using five quantities:
• The temperature T
• The pressure p
• The average volume-integrated vorticity Vor ave (over the entire vo-
lume of THAI+)
1
Vorave =
V
∭ (ωx2 + ωy2 + ωz2 )1/2dV (30)
• Two surface-integrated mass flows over two surfaces in the upper
and lower tubes
ṁ tube = ∬ mdȦ (31)
Temperature, pressure and ṁ tube were selected due to their im-
portance in the validation of CFD codes in nuclear applications. The
choice of the integral quantity Vorave is based on our previous works
where it has been found that the flow field in THAI+ is characterized by
the formation of a high amount of vortices. The quantification of these
vortices can be an important aspect of this natural convection flow. In
total, the considered quantities have been evaluated in 544 points in 9
to 10 different simulation times, so that one could cover all flow re-
gimes in this simulation, including the high turbulent regime at the
beginning of the simulation and the low turbulent regime at the end. In
contrast to our previous works, wall temperature points were also in-
cluded in this study since important flow phenomena occur in the near-
wall region such as the heat transfer process and the vortex formation.
Fig. 1. CAD model of the THAI+ test facility (left); Initial
and boundary conditions in the simplified 3D geometry
which consists of significant details for CFD simulations
(internal installations, etc.) (right).
A. Mansour, E. Laurien
In total, six grids were used to perform the GCI study, see Table 2.
The coarsest grid has approximately 0.56·106 elements while the finest
grid is composed of approximately 39.73·106 elements. The six grids in
this study were generated using refinement ratios greater than 1.3
(Celik et al., 2008). When performing a mesh independence study for
explicit schemes, one should also take the CFL stability condition into
consideration.
u. Δt
CFL = ⩽ 1.
h (32)
With successive grid refinement, h will decrease. In this case, very
large CFL numbers can be reached and this could affect the stability of
the numerical method. The CFL condition in Eq. (32) is an important
criterion especially for the explicit schemes since it ensures in addition
to stability more numerical precision. Although Ansys CFX is an implicit
code, it would be meaningful to achieve CFL numbers which are not too
large since this will affect the numerical precision of the calculations. In
this context, two grid convergence studies were performed. In the first
one, the time step was kept constant while in the second study, a uni-
form CFL number was achieved.
By having a constant time step, the time discretization error will not
be considered and only the spatial discretization error will be quanti-
fied; see for example (Weinman et al., 2006). However, this time step
should be chosen on the basis of the finest grid requirements. Indeed,
the fine grid solution requires the finest time step. So if the time step
independence and numerical stability are achieved on this grid, the
time independence and stability will be also ensured for the coarser grid
solutions. For this reason, a time step sensitivity study was performed
on the finest grid 1 with 39.73·106 elements. Starting from an initial
value Δt = 1 s , the time step Δt was refined to 0.5 s and 0.25 s. Fig. 3
shows the results of this study in two pressure and two temperature
measurement points MP1 and MP2. The temperature profiles show no
significant changes with successive halving of Δt to 0.5 s and 0.25 s.
However, the pressure profiles in the simulation with Δt = 1 s (green
curves) show an important deviation compared to the other profiles
with Δt = 0.5 s and Δt = 0.25 s (from approximately 400 s), which have
almost the same trend. For this reason, the time step was set to
Δt = 0.5 s in the GCI study, since a further refinement to Δt = 0.25 s
does not have any significant influence on the simulation results and
will merely lead to more computational efforts. Nevertheless, it is
meaningful to check the CFL achieved in each grid since a mesh re-
finement with a uniform time step can result in very large CFL numbers.
Even if CFX uses an implicit numerical method, high CFL values can
affect the numerical precision. The evaluation of the volume-averaged
CFL over time for the different grids showed that those were kept in a
small range, see Table 3. Indeed, the highest maximum volume-aver-
aged CFL is equal to 3.504 on the fine grid no. 1.
225
Nuclear Engineering and Design 326 (2018) 220–233
Fig. 2. Inside view of the grid in THAI+. This
consist of tetrahedrons in the flow volume and
prism layers on the structure walls for a better
resolution of the boundary layer.
In the second study, the CFL number was kept uniform by a si-
multaneous grid and time step refinement. The estimation of the exact
solution in the Grid Convergence Index Method is based on a power
series expansion, i.e. the Richardson extrapolation. When a fixed time
step is used, the extrapolated solution will contain the time dis-
cretization error and would not correspond to the exact solution. For
this reason, it would be meaningful to refine the grid and time step Δt
simultaneously. Table 3 shows the values of Δt which were chosen
based on the refinement ratios in Table 2 with the aim of achieving a
uniform CFL value.
The results of the GCI study for the two grid refinement approaches,
i.e. the first one with a fixed time step and the other one with a uniform
CFL, are presented in Figs. 4 and 5. No significant differences between
these two approaches can be observed. Indeed, not only the same trends
but also similar uncertainty and GCI values are detected in the con-
sidered variables. As stated above, the results of the time step study
showed that the fine grid solutions are already time step independent
for Δt = 0.5 s . A further refinement of Δt = 1 s will not have any in-
fluence on the simulation results. For this reason, keeping a uniform
CFL number by successively refining this time step (Δt = 0.5 s) will lead
to similar results as the calculations with a fixed time step and will only
result in more computational efforts. For the vorticity and mass flow in
the REM approach, similar relative errors were also detected. The dif-
ferences between the GCI values in these two quantities are mainly due
to the larger accuracy orders obtained in the study with a uniform CFL
number. The results of the GCI studies in Figs. 4 and 5 also show that
the standard REM approach provides very low and unrealistic error
values compared to the BFM and LSQ approaches. Indeed, this method
led to very large values of the observed order p which are higher than
the maximum theoretical order of CFX pmax = 2. For the study with a
fixed time step, averaged p values of approximately 4.010, 6.185, 3.496
Table 2
Element numbers, averaged grid spacing and refinement ratios of the six grids used in the
GCI study.
Grid Vmesh [m3] Number Elements hi
Grid 6 78.610 563,749 0.052
Grid 5 1,268,908 0.040
Grid 4 3,020,178 0.030
Grid 3 7,118,431 0.022
Grid 2 16,856,294 0.017
Grid 1 39,731,271 0.013
r 21 1.331
r 32 1.333
r 43 1.331
r 54 1.335
r 65 1.307
A. Mansour, E. Laurien
Fig. 3. Time step sensitivity analysis. No significant changes are detected with time step refinement from 0.5 s to 0.25 s.
Table 3
First line: the maximum volume-averaged CFL values in the study with a fixed time step
Δt. Second line: the time steps used in study 2, in which a uniform CFL number is
achieved.
Grid 6 Grid 5 Grid 4 Grid 3 Grid 2
Study 1: Δt=const. CFLmax 1.069 1.105 1.167 1.623 2.709
Study 2: Δt 0.5 s 0.4 s 0.3 s 0.25 s 0.2 s
CFL = const.
and 5.412 were obtained for the temperature, pressure, mass flows and
vorticity. This issue has been also encountered in previous CFD calcu-
lations which are characterized by complex geometries and highly
turbulent flows. The main contributors of this so called noisy grid
convergence could be the lack of the similarity of the unstructured grids
and the use of damping functions and switches in the turbulence models
(Eça and Hoekstra, 2014). The REM approach does not seem to be
appropriate for the error quantification in this complex natural con-
vection flow field since it cannot handle the cases of noisy grid con-
vergence (Celik et al., 2008). The high accuracy orders p will result in
very low GCI values and this will significantly underestimate the spatial
discretization error.
For this reason, the Blend Factor Method BFM has been introduced
in order to provide reasonable values of the accuracy order p. For our
studies, the blend factors β of the advection scheme were obtained
using a worst-case scenario. Indeed, β values were determined at the
beginning of the simulation (t = 20 s), since in this time the flow field is
most turbulent. Consequently, the transport equations are solved using
the lowest accuracy orders, which will provide conservative error es-
timates. Using Eqs. (12) and (13), the evaluation of β resulted in
averaged accuracy orders p of 1.453 and 1.458 for the studies with a
fixed time step and uniform CFL number.
The results of the GCI study using these accuracy orders are also
illustrated in Figs. 4 and 5. The plots for the different quantities show
that the GCI values are significantly more conservative than those
provided by the REM method. Besides, the grid convergence is also
characterized by a non-monotonic behavior. In fact, with successive
Nuclear Engineering and Design 326 (2018) 220–233
grid refinement from mesh 6 to 5 and 5 to 4, the GCI values mostly
decrease. However, the further refinement to mesh 3 with 7.11·106
elements results in a significant increase in the GCI values (7.88% for
the temperature, 1.55% for the pressure, 7.98% for the mass flows and
Grid 1 46.92% for the vorticity). The reason for this behavior may be the
numerical resolution of vortices on mesh 3 with 7.11·106 elements,
3.504
which could not have been resolved on the coarser meshes 4, 5 and 6.
0.15 s
The large GCI value of more than 46% in the volume-averaged vorticity
for the grid transition 4–3 can confirm this statement since this integral
quantity quantifies the amount of vortices over the entire flow field.
The comparison of temperature profiles in Fig. 6 agrees with the results
of the GCI study. Indeed, there is a significant deviation in the data for
the mesh transition 4–3 while a further mesh refinement does not bring
any important changes. Fig. 7 gives an overview about the velocity field
at the top of THAI vessel at 120 s on mesh 3 with 7.11·106 elements. The
flow field is characterized by a high degree of turbulence due to the
vortices and rotational flows which are formed around the geometric
curvatures at the upper part of THAI in both axial and radial directions.
Since the BFM method is also based on a single term series expansion,
i.e. the Richardson Extrapolation with a fixed order p, it does not seem
able to handle this case of noisy grid convergence which requires a
power series expansion with more than one term (Eça, 2017). However,
the BFM method provides more realistic accuracy orders than the
standard REM method and thus gives reasonable and more conservative
error estimates.
This non-monotonic grid convergence was “smoothed” in the Least
Squares Method since this method is based on a power series expansion
with at least two terms, i.e. first and second order. The negative, the
very high and the very low order values will be discarded. The error
bars for T, p, Vorave and ṁ tube in Figs. 4 and 5 show that the uncertainty,
which corresponds to the GCI in the REM and BFM methods, decreases
monotonically with successive grid refinement. In addition, the LSQ
method provides error estimations, which are more conservative than
the BFM method. For instance, simulations on the finest grid 1 with
39.731·106 elements have a numerical discretization error (uncertainty)
of approximately 9.23%, 1.02%, 63.94% and 9.72% for the tempera-
ture, pressure, volume-integrated vorticity and the mass flows,
226
A. Mansour, E. Laurien
Fig. 4. Estimation of the spatial discretization error (Uncertainty for LSQ and GCI for BFM and REM) for the quantities: temperature and pressure.
respectively. The contribution of the standard deviation of the fit in the
LSQ method can be a reason for this behavior (Eqs. (28) and (29)).
Indeed, the fit standard deviation was taken into consideration for the
uncertainty estimation, while only the error values were considered in
the BFM and REM methods. Table 4 shows the average p values and the
distribution of the number of cases in dependence of the observed order
values in the LSQ method. Most of the errors are quantified using an
error estimator with first or second order (p > 2) . For the temperature,
this has been reached in 99.19% of the cases. For the pressure, a p value
higher than two was achieved in 60% of the cases while it was in 40%
of the cases in the range between 0.5 and 2, i.e. the error estimator in
Eq. (14) was used. Table 4 shows also that there is no cases in which p
values lower than 0.5 were reached.
Based on these results of the present Grid Convergence Index study,
it can be concluded that the standard REM approach is not applicable to
this high complicated natural convection flow, since it provides un-
realistically high accuracy orders, which results in a significant un-
derestimation of the spatial discretization error. These high accuracy
order values may also be a sign that the asymptotic range is not reached
even using the finest grids with 39.731·106 elements. In contrast, The
BFM approach provides more reasonable accuracy order values.
However, this method is not able to deal with the noisy grid
227
Nuclear Engineering and Design 326 (2018) 220–233
convergence in our simulations. The LSQ approach turned to be the
most adequate approach in this study since it could handle the non-
monotonic grid convergence. Using this method, a monotonic decrease
of the numerical error with successive grid refinement was detected.
4.2. Wall treatment error
The main near-wall treatment approaches in most CFD codes are the
low-Reynolds-number method and the wall-function method. The low-
Reynolds-number method requires high grid refinement levels in the
near-wall region so that the boundary layer is numerically resolved. In
terms of accuracy, this method is most appropriate. However, high
computational efforts are needed and the generation of fine grids in the
near-wall region is mostly a challenging task in complex CFD cases. In
contrast, the wall-function method works with coarse grids in the near-
wall region and, thus, requires less computational efforts. Nevertheless,
this method, which uses empirical formulations, has a lower accuracy
especially on fine grids. In the current investigations, the SST turbu-
lence model in Ansys CFX 16.1 is employed (Menter, 1994). This uses a
blending function which switches between the low-Reynolds-number
and the wall-function methods depending on the dimensionless wall
distance of the first grid node y+ (Vieser et al.).
A. Mansour, E. Laurien Nuclear Engineering and Design 326 (2018) 220–233

Fig. 5. Estimation of the spatial discretization error (Uncertainty for LSQ and GCI for BFM and REM) for the integral quantities: volume-integrated vorticity Vorave and mass flows ṁ tube in
the upper and lower tubes.

+
The idea behind this investigation is to quantify the error resulting Reynolds-number approach is used for yave < 1while wall-function of
+
from the use of the low-Reynolds and wall-function methods. For this, the SST model is employed in the buffer layer (5 < yave < 30) and log
four grids with different y+ are generated. Since transient simulations +
region (30 < yave ) . In the literature, it is also often assumed that a ^ value
+
are computed, the considered yave values are the domain average y+ below 5 is sufficient to numerically resolve the boundary layer, see for
over time example (Blocken et al., 2007). For this reason, grid 4 which has
Nt Nnode
1.269·106 elements and 1 < yave +
= 2.559 < 5 was generated. In this way,
+ 1 ⎡ 1 ⎤ the impacts of the different ranges of the y+ values can be detected. The
yave =
NΔt
∑ ⎢N ∑ y+ ⎥,
j ⎣ node i ⎦ (33) properties of the grids used in our investigations are reported in
Table 5.
where NΔt is the number of time steps and Nnode stands for the number of The same variables as in the GCI study are also considered in this
the first grid nodes near the wall. Grid 1 is composed of 8.222·106 investigation. Those are the gas temperature T, the pressure p, the vo-
elements and has twelve fine prism layers on the walls. This resulted in lume-integrated vorticity Vorave and the mass flows in the upper and
an averaged y+ over time yave
+
= 0.999 < 1. In this case, the boundary lower tubes ṁ tube . The averaged relative errors ε as defined in Eq. (7) are
layer is numerically resolved. Grid 2 has 7.16·106 tetrahedral elements shown in Table 6. The reference values for the calculation of ε are the
+
with 5 < yave =20.795 < 30, i.e. we lie in the transition buffer layer. finest grid solutions (grid 1). As stated above, this grid uses the low-
Grid 3 is composed of 0.204·106 tetrahedral elements and a yave +
value Reynolds-number approach and that is why assumed to provide the
+
yave = 52.32 > 30, which indicates that we are in the log layer where most accurate results.
the logarithmic law is applied. By comparing the solutions on these The high relative errors reported in Table 6 show that the wall
three grids, the wall treatment error is quantified since the low-

228
A. Mansour, E. Laurien Nuclear Engineering and Design 326 (2018) 220–233

Fig. 6. Comparison of the temperature profiles over time in two MP in THAI+ and in line 3 at 240 s for the six grids defined in Table 2 (fixed Δt = 0.5 s ).

treatment approach has a significant influence on the simulation re- pressure, 60.46% in the average volume-integrated vorticity and 8.43%
+
sults. It can be also seen that the higher the yave values, the larger are the in the mass flows. However, keeping yave +
= 2.559 below 5 in grid 4 re-
+
wall treatment errors. Indeed, the comparison of grid 1 with yave = 0.999 duced considerably the relative errors to 2.63%, 0.28%, 11.47% and
+
and grid 2 with yave = 20.795 resulted in relative errors of 9.63%, 1.56% in T, p, Vorave and ṁ tube . Besides, the results in Table 6 and Fig. 8
0.41%, 47.97% and 1.73% in T, p, Vorave and ṁ tube . Except for the show that the near-wall treatment approach has a great influence on the
temperature, the increase of the yave+
values to 52.32, i.e. grid 3, led to a formation of the vortex system in THAI+. As can be seen in Fig. 8, the
general increase in the relative errors. Those are equal to 0.96% in the lower the y+ values, the more vortices are initiated which means that

Fig. 7. Velocity field at 120 s in the upper part of

THAI vessel (left) and in two horizontal sectional
planes: Section 1 at the top pf THAI and Section 2
through the condensate trays and inner cylinder
(right).

229
A. Mansour, E. Laurien Nuclear Engineering and Design 326 (2018) 220–233

Table 4 The first simulations performed in this study are a part of a “double
Average p values using the least squares method and the frequency of the case dis- blind” benchmark of the initial operating test TH 27 of THAI+, i.e.
tributions in percent (Study with constant Δt).
there were no measurement data available as those simulations were
Temperature T Pressure p Vorticity Vorave Mass flows performed. For this reason, most of the boundary conditions were de-
ṁ tube termined based on some assumptions and the pre-data available in that
time. Since all walls in the experiment were completely insulated
average p value 21.53 4.26 55 4.29
through rockwool and aluminum isolation, the structure walls in the
p < 0.5 in% 0 0 0 0
0.5 ⩽ p ⩽ 2 in% 0.81 40 0 15
double blind simulations were set to adiabatic. However, important
p > 2 in% 99.19 60 100 85 heat losses were detected during the experiment. Fig. 10 shows a
comparison of the simulation results using the adiabatic boundary
condition of gas temperature profiles over time in seven monitor points
Table 5 (black curves) with the experimental data in the corresponding mea-
grid properties. surement points MP (red curves). Because the heat transfer was not
modeled in the simulations, Fig. 10 shows a big deviation between the
Grid 1 Grid 2 Grid 3 Grid 4
simulation results and the experimental data. Indeed, the simulated
6
Number of elements in 10 8.222 7.160 0.204 1.269 temperature increases in all MP until it reaches a constant value
+ 0.999 20.795 52.32 2.559
yave (thermal equilibrium) while the measured temperature either decreases
or stay nearly constant.
To model the heat transfer through the walls, the total heat transfer
Table 6 coefficient HTC for each part of THAI+ were used. Since this is the
Relative error ε due to the different wall treatments. The reference value for the calcu-
initial operating test of the facility, the different heat transfer coeffi-
lation of ε is the fine grid solution 1.
cients, especially of the PAD vessel and the pipe connections, are still
Quantity T p Vorave ṁ tube not exactly determined. Thus, we estimated these heat transfer coeffi-
cients based on some assumptions and previous experiment results in
Grids 1 and 2: ε in% 9.63 0.41 47.97 1.73
THAI vessel (Freitag, 2016). Indeed, besides the pipe connections, we
Grids 1 and 3: ε in% 7.20 0.96 60.46 8.43
Grids 1 and 4: ε in% 2.63 0.28 11.47 1.56 considered all other PAD and THAI vessels as pipe shaped structures
with multi-layers (wall structure, oil layer, Rockwool and aluminum
layers), for which we estimated the total heat transfer coefficients using
the thermal conductivity and thickness of each layer. The estimated
heat transfer coefficients for each part of the test facility are shown in
Table 7. Fig. 10 shows the simulation results using those heat transfer
coefficients (green curve).
These simulations were performed on the same grid as the adiabatic
simulation, i.e. on approximately 1.286·106 elements. Fig. 10 shows
that the simulation results with the estimated HTC agree better with the
experimental data since the heat losses are considered. Since the Blend
Factor Method BFM (GCI study) and the mesh independency studies
(Fig. 6) showed that grid 3 with 7.11·106 elements has a reasonable
compromise between numerical accuracy and costs, the further simu-
lations were run on this grid using the HTC defined in Table 7. Fig. 10
shows the results of these simulations, where there is a much better
agreement with the experimental data in almost all measurement
points. Even in MP 2 and 7, where the temperature profiles on the
coarser grids with 1.286·106 elements match better the experimental
data at the beginning, there is a better agreement with the time be-
Fig. 8. comparison of the average volume-integrated vorticity in four grids with different
+
tween the results on the fine grid and the measurements (from 3200 s).
yave values.

these vortices are mostly formed in the near-wall region when the
boundary layer is numerically resolved leading to a highly turbulent
flow field. Fig. 9 shows the three-dimensional flow impressions on grid
1 and 3 by means of forward streamlines in THAI+, where it can be
recognized that the flow field is far more complicated on grid 3 with
+
yave +
= 0.999 than on grid 3 where yave is equal to 52.32.
4.3. Comparison of simulation results with experimental data
After the quantification of the numerical and model error, the si-
mulation results will be compared to the experimental data. In order to
perform a validation process, one needs to quantify besides the nu-
merical and wall-treatment error also the experimental uncertainties.
The measured quantities used for validation are mainly the temperature
and pressure. The measurement uncertainty of the thermocouples used
in the test transients is equal to ± 0.15 K while the pressure measure-
ment instruments have an uncertainty of ± 15 mbar (Kanzleiter et al.,
2007).
4.4. Parallel computing
We carried out the scalability tests on the CRAY XC-40 Hazel Hen of
the High Performance Computing Center Stuttgart HLRS which is a
supercomputer with 7712 compute nodes. Each node is composed of 24
processing cores and has 128 GB memory.
In order to measure the parallel performance of CFX 16.1, the
Speedup is used
tref
Sp = .
tp (34)
here, tref is the reference wall clock time on the reference core number
while tp stands for the wall clock time when using p cores. For grid
partitioning, CFX uses a node-based partitioning method. The default
partitioning algorithm is the Multilevel Graph Partitioning Software
MeTiS (Karypis and Kumar, 1998).
For the scalability tests, we performed part simulations of the nat-
ural convection flow in THAI+ for a physical time t = 150 s. Since our

230
A. Mansour, E. Laurien
Fig. 10. Comparison of the CFD simulation results with the experimental data for different heat transfer approaches and different grids.
goal is to define a kind of Best Practice Guidelines for the optimal use of
computational resources depending on the grid resolution, we per-
formed our tests on four numerical grids with different refinement le-
vels. The fine grid (grid a) consists of approximately 83·106 elements
and 24·106 nodes, while the other grids have 39.731·106 elements and
11.5·106 nodes (grid b), 7.11·106 elements and 1.925·106 nodes (grid c)
and 1.268·106 elements and 0.347·106 nodes (grid d). The results of the
scalability tests are shown in Fig. 11 and Table 8. These show that the
231
Nuclear Engineering and Design 326 (2018) 220–233
Fig. 9. Forward streamlines through the middle
plane of THAI+ at 120 s on grid 1 with 8.222·106
+
elements and yave = 0.999 (left) and grid 3 with
+
0.204·106 elements and yave = 52.32 (right).
more elements/nodes the grid has, the better the parallel performance
is. Indeed, on the fine grids a and b, CFX Speedup is characterized by a
super linear behavior at the beginning (up to 710 cores for grid a and
635 cores for grid b). The maximum speedups on those two grids are
reached at relatively high core numbers of approximately 1838 and
1658, while the maximum achieved core number on grid c and d
amounts to 1100 and 562 cores (see Table 8). The ratio of the maximum
Speedup to the ideal one almost decreases with decreasing number of
A. Mansour, E. Laurien
Table 7
Heat transfer coefficients for the different parts of THAI+. The heat transfer coefficient
of the heated part of THAI vessel was not used in this simulation.
Part
Heat Transfer Coefficient ⎡ 2 ⎤
⎣m K ⎦
THAI heated part 0.845
THAI unheated sump 0.9
Pipe connections 1.011
PAD vessel 0.9
grid elements. This is equal to 74% and 59% on the fine grids a and b
and only 48% and 32% on the coarser grids c and d.
Besides, the finer the grid, the higher the required computational
time is (Table 8). These and the others values presented in Table 8 can
be taken into account in order to determine the optimum use of com-
putational resources in CFD calculations depending on the grid re-
solution.
5. Conclusions
This work presents a step to define Best Practice Guidelines BPG in
terms of the quantification of the spatial discretization error, the wall
treatment error and also the optimum use of computational resources
for CFD simulations in nuclear applications and similar disciplines. For
this, a natural convection flow simulation in the newly constructed two-
room model containment THAI+ was performed using the CFD package
Ansys CFX 16.1.
Firstly, a Grid Convergence index study GCI was performed in order
to quantify the spatial discretization error. For this, three versions of the
GCI were considered. Those are the Standard method REM, the Least
Squares Method LSQ and the Blend Factor Method BFM, which has
been introduced in this work. A total of six grids were used to perform
the GCI study. The finest grid has approximately 39.731·106 elements.
The other grids have 16.856·106, 7.118·106, 3.02·106, 1.268·106 and
Nuclear Engineering and Design 326 (2018) 220–233
0.563·106 elements.
Using these grids, two mesh refinement approaches were compared.
The first approach was performed with a fixed time step based on the
W
fine grid requirements while the second one with a uniform CFL
number. Almost no differences between these two approaches were
observed since not only the same trends were identified but also similar
error estimates were detected.
In addition, the results of the GCI studies showed that the standard
REM method provides unrealistically high accuracy order values due to
the high complexity of the flow field and the non-similarity of the un-
structured grids used in this CFD simulation. These high order values
can be, however, a sign that the asymptotic range is still not reached
even using very fine grids with 39.731·106 elements.
The BFM approach gives more reasonable values of the accuracy
order and, thus, more realistic error estimates. Nevertheless, the BFM
method could not handle the noisy grid convergence in this highly
turbulent natural convection flow field since it is based on a one term
series expansion, i.e. the Richardson extrapolation with a uniform ac-
curacy order. This noisy grid convergence was nevertheless handled by
the LSQ method which uses a power series expansion with more than
one term. However, and even on the finest grids with 39.731·106 ele-
ments, high numerical uncertainties were detected using the LSQ
method. This can be a sign that the asymptotic range is still not reached
or that reaching the asymptotic range in this complex configuration and
flow field is simply not possible. Nevertheless, the error estimates
quantified in the LSQ study can be taken into account for the estimation
of the numerical error in future CFD simulations in THAI+ such for
example, model validation.
For the quantification of the wall treatment error, four grids with
+
different average dimensionless wall distances yave were considered.
+
These grids have yave values of 0.999, 2.559, 20.795 and 52.32. The
comparison of the calculations using these grids ensures the detection
of the impacts of the different ranges of y+ values on the CFD calcu-
+
lations. The results of this investigation showed that the larger the yave
values, the higher are the wall treatment errors and the lower is the
Fig. 11. comparison of the ideal and CFX speedup
on grids with different numerical resolutions.
232
A. Mansour, E. Laurien
Table 8
Comparison of the scalability behavior on four numerical meshes with different resolutions (see Fig. 11) in the maximum achieved speedups Spmax .
At the maximum Spmax Grid a
Reference Core Number 120
Spmax / Spideal 11.4/15.32 = 0.74
Core number 1838
Grid Elements/Core 46151
Grid Nodes/Core 13373
tphy → tcomputational 1 s → 44 s
amount of vortices. Indeed, most vortex systems are formed in the near-
wall region so if the boundary layer is numerically resolved, this will
also result in the resolution of more vortices leading to a highly com-
plex flow field.
The scalability tests using four grids with different refinement levels
showed that the maximum speedup, and hence the optimal core
number, depends significantly on the used grid elements/nodes. The
more elements the grid has, the more efficient is the parallel computing
but -obviously- the bigger the required computational time.
Acknowledgments
This work was supported by the German Federal Ministry of
Economic Affairs and Energy (BMWi) on the basis of a decision by the
German Bundestag, project number 1501493. The THAI project was
carried out on behalf of the Federal Ministry for Economic Affairs and
Energy (project no. 1501455) on the basis of a decision by the German
Bundestag.
References
Babić, M., Kljenak, I., Mavko, B., (Ed.), “Simulation of Atmosphere Mixing and
Stratification in the THAI Experimental Facility with a CFD Code”, Nuclear Energy for
New Europe, Bled, Slovenia, September 5-8, 2005.
Barth, T.J., Jespersen, D.C., “The Design and Application of Upwind Schemes on
Unstructured Meshes”. In: 27th Aerospace Engineering Meeting, Nevada, USA,
January 9-12, 1989.
Blocken, B., Stathopoulos, T., Carmeliet, J., 2007. CFD Simulation of the Atmospheric
Boundary Layer: wall function problems. Atmos. Environ. 41, 238–252.
Carozza, A., 2016. Numerical investigation of heat exchanger performances for a two
engine aircraft in pusher configuration by an U-RANS code. Appl. Therm. Eng. 99,
1048–1056.
Celik, I.B., Ghia, U., Roache, P.J., Freitas, C.J., Colemann, H., Radd, P.E., 2008. Procedure
for estimation and reporting of uncertainty due to discretization in CFD applications.
J. Fluids Eng. 78001.
Eça, L., 2017. Private Communication. Instituto Superior Técnico, Lisbon, Portugal.
Eça, L., Hoekstra, M., 2014. A procedure for the estimation of the numerical uncertainty
of CFD calculations based on grid refinement studies. J. Comput. Phys. 262, 104–130.
Freitag, M., Schmidt, E., Gupta, S., Colombet, M., Kühnel, A., Von Laufenberg, B., March
2016. Technical Report: Commissioning Test of the Two Vessel Configuration Test
TH.-27. Becker Technologies GmbH, Eschborn, Germany.
Freitag, M., Private Communication, Becker Technologies GmbH, 2016.
Günther, C., Private Communication, Ansys Germany GmbH, 2016.
Nuclear Engineering and Design 326 (2018) 220–233
Grid b Grid c Grid d
120 24 24
8.15/13.82 = 0.59 22.23/45.84 = 0.48 7.5/23.42 = 0.32
1658 1100 562
23963 6471 2257
6936 1750 618
1 s → 49.17 s 1 s → 15.6 s 1 s → 7.5 s
Houkema, M., Siccama, N.B., Lycklama à Nijeholt, J.A., Komen, E., 2008. Validation of
the CFX4 CFD code for containment thermal-hydraulics. Nucl. Eng. Des. 238,
590–599.
IAEA, “Use of computational fluid dynamics codes for safety analysis of nuclear reactor
systems: Summary report of a technical meeting”, Pisa, Italy, 11-14 November 2002”,
2003.
Kainz, M., Private Communication, Ansys Germany GmbH, 2016.
Kanzleiter T., Ahrens G., Fischer K., Häfner W., Kühnel A., Poss G., Funke F., Langrock G.,
Allelein H.J., Weber G., Schwarz S., Final Report: Experimental Facility and Program
for the Investigation of open Questions on Fission Product Behavior in the
Containment, ThAI Phase II“, Becker Technologies GmbH, Eschborn, Germany,
March, 2007.
Karypis, G., Kumar, V., 1998. A fast and high quality multilevel scheme for partitioning
irregular graphs. SIAM J. Sci. Comput.
Kwaśniewski, L., 2013. Application of grid convergence index in FE computation. Bull.
Polish Acad. Sci. Tech. Sci. 61.
Lenarcic, M., Eichhorn, M., Schoder, S.J., Bauer, Ch., Numerical Investigation of a High
Head Francis Turbine Under Steady Operating Conditions using Foam-Extend”,
Journal of Physics, Francis-99 Workshop 1: Steady Operation of Francis Turbines,
Conference Series 579, 2015.
Longest, P.W., Vinchurkar, S., 2007. Effects of mesh style and grid convergence on par-
ticle deposition in bifurcating airway models with comparisons to experimental data.
Med. Eng. Phys. 29, 350–366.
Mansour A., Laurien, E., “Simulation of a Natural Convection Flow with Humid Air in a
Two-Room Geometry”, Computational Fluid Dynamics (CFD) for Nuclear Reactor
Safety Applications (NRS), Boston, MA, USA, September 12–15, 2016.
Menter, F.R., 1994. Two-equation eddy-viscosity turbulence models for engineering ap-
plications. AIAA-J. 32 (8), 1598–1605.
NEA/CSNI, “International Standard Problem ISP-47 on Containment Thermal Hydraulics,
Final Report”, NEA-CSNI/R (2007) 10, 11. September, 2007.
Pereira, F.S., Eça, L., Vaz, G., “On the Order of Grid Convergence of the Hybrid
Convection Scheme for RANS Codes”, Congress on Numerical Methods in
Engineering, Bilbao, Spain, June 25-28, 2013.
Roache P.J., “Verification and validation in computational science and engineering”,
Hermosa, Albuquerque, NM, 446 pp., 1998.
Vieser, W., Esch, T., Menter, F., Smirnov, P., Heat Transfer Predictions Using Advanced
Two-Equation Turbulence Models“, Ansys Germany and New Technologies and
Services, Germany/Russia.
Weinman, K.A., Van der Ven, H., Mockett, C.R., Knopp, T.A., Kok, J.C., Perrin, R.T.E.,
Thiele, F.H., Study of Grid Convergence Issues for the Simulation of the Massively
Separated Flow Around a Stalled Airfoil Using DES and Related Methods”, In:
European Conference on Computational Fluid Dynamics ECCOMAS CFD, Delft, The
Netherlands, 2006.
Zhang, J., Laurien, E., 2015. 3D numerical simulation of flow with volume condensation
in presence of non-condensable gases inside a PWR containment. Springer
International Publishing, Cham, pp. 479–497.
Zirkel, A., Doebbener, G., Laurien, E., “CFD Simulation of Forced Flow Within the THAI
Model Containment”. In: 17th International Conference on Nuclear Engineering,
Brussels, Belgium, 2008.
233