Science of the Total Environment 695 (2019) 133801

Contents lists available at ScienceDirect

Science of the Total Environment

journal homepage: www.elsevier.com/locate/scitotenv

Review

Effects of airflow on the thermal environment and energy efficiency in

raised-floor data centers: A review
Chaoqiang Jin a,b, Xuelian Bai a,b,⁎, Chao Yang c
a
Key Laboratory of the Three Gorges Reservoir Region's Eco-Environment, Ministry of Education, Chongqing University, Chongqing 400045, China
b
National Centre for International Research of Low-carbon and Green Buildings, Chongqing University, Chongqing 400045, China
c
Beijing Rainspur Tech Co., Ltd, Beijing 100000, China

H I G H L I G H T S G R A P H I C A L A B S T R A C T

• Analyzes the multi-scale effects of air-

flow on thermal environment in data
centers
• Summarizes the simulation tools and
presents the key issues and main fea-
tures of CFD simulation
• Summarizes and analyzes the character-
istics of the existing performance met-
rics of air distribution
• Presents the solutions and optimiza-
tions on airflow distribution for cooling
energy saving
• Future research shall make great efforts
to achieve energy saving through man-
aging airflow distribution.

a r t i c l e i n f o a b s t r a c t

Article history: Airflow is crucial for air-cooled data centers. Its flow path and distribution influences the thermal environment
Received 26 April 2019 and energy efficiency of raised-floor data centers. This paper provides a review of the topic including the aspects
Received in revised form 1 August 2019 of airflow factors, numerical study, airflow performance metrics, and thermal optimization. Based on the multi-
Accepted 5 August 2019
scale characteristics of the data center, the thermal environment is categorized into room-level, rack-level, and
Available online xxxx
server-level environments. For the room-level thermal environment, the main factors include layout, raised
Editor: Deyi Hou floor plenum and ceiling height, and perforated tiles. For the rack level, the effects of the porosity ratio of rack
door, airflow rate/temperature, server population, server arrangement and power density are considered. For
Keywords: the server level, airflow rate and server fan speed are investigated. Moreover, numerical studies have been widely
Data center employed to understand the thermal environment of data centers. The selections of simulation tool and the
Thermal environment methods for simplifying and validating the models are key to predicting the data center's thermal behavior cor-
Simulation rectly. In addition, airflow performance metrics and multi-scale thermal optimization are summarized and
Performance metrics discussed. This review aims to emphasize the importance of the airflow in data centers and thus serve a guiding
Optimization
reference for airflow design and energy efficiency in data centers. Some recommended topics for future research
are also provided.
© 2019 Elsevier B.V. All rights reserved.

⁎ Corresponding author at: Key Laboratory of the Three Gorges Reservoir Region's Eco-Environment, Ministry of Education, Chongqing University, Chongqing 400045, China.
E-mail address: xuelianbai@cqu.edu.cn (X. Bai).

https://doi.org/10.1016/j.scitotenv.2019.133801
0048-9697/© 2019 Elsevier B.V. All rights reserved.
2 C. Jin et al. / Science of the Total Environment 695 (2019) 133801

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Factors influencing the thermal environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. Room level. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1.1. Layout of CRACs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1.2. Raised floor plenum and ceiling height . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.3. Aisle containment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.4. Perforated tiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2. Rack level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.1. Porosity ratio of rack door . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.2. Airflow rate and inlet air temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.3. Server population, arrangement, and power density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3. Server level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3. Numerical study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Simulation software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2. Numerical simulation and validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.1. The model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.2. Mesh generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.3. Boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.4. Turbulence model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.5. Experimental validation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4. Airflow performance metrics and thermal optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.1. Performance metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.2. Optimization of airflow distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2.1. Room level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2.2. Rack level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2.3. Server level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5. Conclusion and recommendations for future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1. Introduction
With the rapid development of information and intelligence, large-
scale data centers have been constructed around the world. According
to the Synergy Research Group, there were nearly 400 large-scale data
centers in the world in 2017. Among these, 44% were in the United
States, 20% were located in China, Japan, and the United Kingdom, and
3–5% were in Australia, Germany, Canada, Singapore, India, and Brazil
(www.199it.com/archives/670459.html). However, due to the dramatic
growth of the IT industry, the energy consumption of data centers has
been increasing rapidly. In 2017, data centers consumed about 2% of
the world's total electricity consumption, and it has been estimated
that this proportion will reach 5% by 2024 (C.A.R, 2019).
Data centers consist of four major components: power equipment,
cooling equipment, IT equipment, and miscellaneous component loads
(Dai et al., 2014). IT equipment is used for data management, process-
ing, storage, and routing, and the other three components are used to
maintain the operational reliability of the IT equipment. These compo-
nents account for 10%, 40%, 45%, and 5% of the energy consumption, re-
spectively (C.A.R, 2018; Committee, 2017). Thus, most the energy is
consumed by the IT equipment and cooling systems. Considering the
numerous energy consumptions of a data center, many advanced
energy-saving technologies have been developed and employed to re-
duce the energy consumption of these two parts of the system. Energy
efficient technologies for IT equipment can be divided into two aspects:
efficient hardware and efficient software. For efficient hardware, heat
reduction methods such as dynamic voltage and frequency scaling are
typically utilized for the information communications technology.
These methods reduce the frequency/voltage of central processing
units (CPUs), which leads to reductions of chip-level power consump-
tion (Dai et al., 2014). Additionally, using multi-speed disks with
smaller spin-up and spin-down times can reduce the disk energy con-
sumption required for server workloads (Carrera et al., 2003;
Gurumurthi et al., 2003). For efficient software, the virtualization ap-
proach can be used to reduce the utilization of servers. This technology
can optimize the use of servers and shut down underutilized servers,
which can bring an emissions reduction of up to 27% (Webb, 2008).
Moreover, heat pipe (Ding et al., 2016; Yue et al., 2018) and on-chip
cooling (Marcinichen et al., 2012; Marcinichen and Thome, 2012) are
energy-saving technologies for server cooling. Compared with tradi-
tional air cooling, the heat pipe can reduce the cooling energy required
by 38.9% (Ding et al., 2016) and on-chip cooling may achieve a 60% en-
ergy saving for blade server boards (Marcinichen et al., 2012). In addi-
tion, a proper cooling system, which removes the heat converted from
the electrical power, is important for data centers. A suitable cooling
system should achieve high-energy efficiency, low-cost, and reliability,
and it should be sufficient for the worst-case scenario. For a best result,
the cooling system effectiveness index should surpass 2.5 (Ni and Bai,
2017). Traditional cooling systems normally consume approximately
50% or more of the energy of the whole data center infrastructure.
Free cooling, such as air-side and water-side economizers, can help re-
duce energy costs (Daraghmeh and Wang, 2017; Ebrahimi et al., 2014;
Hassan et al., 2015; Zhang et al., 2014a). The application of such cooling
technologies could bring electronic power savings of 2.8%–46% (Bao
et al., 2012; Han et al., 2013; Lee and Chen, 2013). In particular, an inte-
grated system of mechanical refrigeration and thermosiphoning can
bring an energy savings of up to 47.3% (Zhang et al., 2015). Moreover,
a hybrid free-cooling system with a latent heat thermal energy storage
unit using a predictive control method can achieve an energy savings
of up to 18% (Wang et al., 2018). In addition, the cooling efficiency ben-
efit provided by airflow distribution has also been a focus of energy
management in data centers. Row-based/in-row cooling is installed in
rack rows, which shortens the distance of air flow from computer
room air conditioning units (CRACs) to racks. Thus, it reduces the cold
loss of air flow in the transportation process. The heat pipe backboard
air conditioner is a rack-level cooling strategy with airflow completely
 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
unaffected by any installation changes or engine room constraints. Thus
the rated cooling capacity of the CRAC can be used fully (Dunlap and
Rasmussen, 2019; Liu et al., 2015).
To illustrate the relationship between energy and thermal environ-
ment, Fig. 1 depicts the power flow and heat flow for a typical data cen-
ter. As shown in the figure, the IT equipment is the end of the power
flow and also the beginning of the heat flow. The power consumption
of the IT equipment determines the amount of cooling load of the data
center rooms. Air is the medium through which heat is dissipated
from the chip level, through the server level, to the rack level, and to
the room level, and then eventually dissipated through the refrigeration
cycle. At the same time, the cooling system provides the work condition
for the IT equipment and is designed to prevent hot spots. If the airflow
distribution does not meet the working conditions for the IT equipment,
it is bound to increase the burden of the cooling system and thus affect
its efficiency. According to recent literature, the potential gains in the
coefficient of performance of a data center system (COP) that is the
ratio of heat dissipation of IT equipment to the energy consumption of
cooling system, have been evaluated at +0.15 (approximately 8%) for
every 5 °C increase in air temperature (Breen et al., 2010). In addition,
the inlet air temperature of the servers has an impact on their energy
consumption because the power leakage of the chip is a function of
this environmental factor. The power consumption increased N80 W
over the temperature window from 17 °C to 35 °C (Moss and Bean,
2010). Although this effect is small for one server, it is large for an entire
data center. Thus, the thermal environment is crucial for not only the
cooling efficiency of the cooling system but also for the energy saving
of the servers.
At present, although there are many ventilation configurations in
data centers (Lu et al., 2018), this paper only focuses on raised-floor
supply, and the current literature review provides an overview of previ-
ous work on the effects of airflow on the thermal environment of a data
center from the room level to the server level. The first part of this re-
view includes an introduction to the state of the art of data center
energy-saving strategies, with an emphasis on the effect of airflow dis-
tribution in energy efficiency. In consideration of the formation of the
thermal environment, multi-scale factors influencing the thermal envi-
ronment are discussed in Section 2. Next, simulated tool selection and a
discussion about ways to simplify and validate models for thermal
Fig. 1. Power flow and heat flow in data centers (power flow in the solid frame and heat flow in the dotted frame).
3
environment simulation are presented in Section 3. Then, in Section 4,
attempts to evaluate airflow distribution and thermal optimization for
energy efficiency are reviewed and summarized. Finally, Section 5 con-
cludes the article with a set of reflections acquired by this work and fur-
ther research opportunities. It is hoped that this paper will be useful for
designers, engineers, operators, and researchers of data centers tasked
with improving the thermal environment, achieving appreciable energy
efficiency, and eventually realizing socially sustainable development.
2. Factors influencing the thermal environment
In the data center, it is often the case that a poor cooling effect, pre-
mature failure of servers, and poor reliability caused by a poor thermal
environment result in significant increases in energy consumption and
operating costs. Thus, it is apparent that good airflow may promote en-
ergy savings and lower operating costs and that the thermal environ-
ment is particularly critical to data centers. Because air delivery is a
multi-scale problem, in which the pore size of floor tile or the floor
gap is on the order of millimeters, while the rack size or room size is
on the order of meters (Arghode and Joshi, 2016), this part of the
paper reviews the effects of various factors on the thermal environment
from the room level to the server level.
2.1. Room level
2.1.1. Layout of CRACs
Airflow distribution, as a key factor of the indoor air environment, is
a complex function of the positions of racks and CRACs. At present,
raised-floor air supply is used widely in data centers. Cold air comes
from the CRACs and then passes through perforated tiles to cool the
racks. Hence, the uniformity of flow rates is firstly influenced by the po-
sition of CRACs. The common designs of CRAC position are summarized
in Fig. 2. The air outlet direction of the CRACs can be parallel to or per-
pendicular to the row of perforated floor tile as a consequence of the
placement of the CRACs, and the CRACs may be located on either one
side or two sides of the end of cold aisles. Different CRAC layouts result
in a different thermal environment of the data center room.
When the CRACs are located on one side, the airflow rate of the per-
forated floor tile increase along the airflow direction. However, the
4 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
Fig. 2. The common CRAC position in data centers.
airflow of tiles perpendicular to CRACs is more uniform than that of tiles
parallel to CRACs. This is true because tiles perpendicular to CRACs have
the same distance from the CRACs, which results in a same pressure per-
pendicular to the airflow direction. Conversely, the pressure for tiles
parallel to CRACs increases along the airflow direction, leading to a max-
imum airflow rate away from the CRACs. When the CRACs are located
on two sides, as shown in Fig. 2(a) and (c), the air flow rate is symmetric
and maximum around the middle rack, and reversed flow occurs in a
few perforated tiles near the CRACs (Nada et al., 2016b; Rambo et al.,
2007) due to the distribution of low pressure (Nagarathinam et al.,
2013). Additionally, the rack inlet air temperature always increases as
a result of short-circuiting of the chilled air, which returns directly
back to the intakes of the CRACs (Schmidt and Iyengar, 2005). The air-
flow distribution is also a strong function of the air direction of the
CRACs in operation. Especially, when both CRACs discharge air in the
same direction, as in Fig. 2(b) and (e), the airflow distribution is similar
to that of CRACs on one side, but it will be improved by installing an out-
ward scoop in the middle of the CRACs (Schmidt et al., 2004). In addi-
tion, locating the CRACs units perpendicular to the racks row (see
Fig. 2(d)) not only enhances the uniformity of the air flow from the per-
forated tiles along the rack row but also reduces hot air recirculation at
the end racks and cold air bypass at the middle racks of the row (Nada
and Said, 2017).
Considering the effects of layout of the CRACs on airflow distribution,
to produce a good air distribution, airflow perpendicular to the perfo-
rated tile rows seems to be the best choice. Additionally, the high-
density racks should be arranged in the rows away from the CRACs.
However, from the perspectives of arrangement of the space, ease of
maintenance, and sense of aesthetics, separating the rack rows and
CRACs into two regions is the best option. Hence, CRACs located at the
ends of each rack row could be the optimal choice, and, due to the air-
flow distribution, the power density for a rack row is suggested to
have a normal distribution.
2.1.2. Raised floor plenum and ceiling height
The static pressure in the plenum is the power source facilitating the
airflow passing through the perforated tiles. This static pressure is
mainly influenced by the height of the plenum. As the height is in-
creased, the airflow in the horizontal planes weakens, leading to re-
duced non-uniformity in the pressure distribution and therefore in the
flow rate through the tiles (Schmidt et al., 2001). This occurs because
the velocity in the plenum decreases and the static pressure increases
as the height increases, making it easier for airflow to exit. However,
there is a limit to the benefit of higher plenum height. There is no re-
verse flow at a plenum height of 0.76 m (30 in.) (Karki et al., 2003;
Patankar, 2008), but there is no great benefit to increasing the plenum
depth beyond 0.61 m (24 in.) (VanGilder and Schmidt, 2005).
Normally, in the raised plenum, there are some chiller pipes, ca-
blings, and wires, which affect the airflow rate and increase the resis-
tance from the air supply outlet to the perforated tiles. These
obstructions can be divided into four types: horizontal chilled water
pipes, vertical chilled water pipers, hot aisle trays and cold aisle trays
(VanGilder et al., 2012). Considering that the static pressure of the fan
is a function of the resistance coefficient and the square of the air flow
rate, when the static pressure of the fan is constant, the obstructions
under the raised plenum increase the resistance coefficient and conse-
quently reduce the total CRAC flow rate. According to Bhopte et al.
(2006c), these reductions account for 1.2–14.4% of the total CRAC flow
rate. Additionally, reduced flow rates inevitably result in higher inlet
temperatures because the starved servers draw hot exhaust due to the
lack of cooling air. This also increases the occurrence of hot spots
(Fulpagare et al., 2015). To avoid these circumstances, under-the-
floor-obstructions should be placed far from perforated tiles and
CRACs. It is better if smaller diameter pipes and trays are placed side-
by-side (Bhopte et al., 2006d), and the height of obstructions is not
more than one-third of the plenum depth (VanGilder et al., 2012).
From the above comprehensive consideration of the great benefit of ple-
num height and the obstructions under the plenum, data center ple-
nums with a height range of 0.76–0.91 m (30–36 in.) are the best
option (Beitelmal, 2016).
Ceiling height, especially for the open-aisle data center, is another
factor affecting hot air recirculation and cold air bypassing, and thus
influencing indoor air temperature. With increase of ceiling height,
more hot air is recirculated, which affects the rack inlet temperature.
 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
At lower ceiling height, the hot air does not have sufficient space to
move and mix with the cold air, which results in heat accumulation.
This accumulation is decreased with height increase. With sufficient in-
crease, hot air and cold air can mix well, resulting in an increase in the
cold supply air temperature, and then the exhaust air temperature.
Hence, it is suggested that the optimum ceiling height is 2.52 m
(Bhopte et al., 2006a; Nagarathinam et al., 2013). However, if there
are no major issues installing electrical/communication cables above
the rack server, the ceiling height may be reduced with no major effects
on airflow efficiency. (Cho et al., 2014).
2.1.3. Aisle containment
Containment is an effective method for keeping the hot air from
flowing into the cold aisle. In other words, it can minimize the air tem-
perature increase from the outlet of the CRAC to the inlet of the servers.
Containing air is an important energy-saving strategy for data center
optimization. There are two types of air-containment systems: semi-
enclosed containment (side/door containment or top/roof contain-
ment) and fully enclosed containment (side & top containment) sys-
tems, as shown in Fig. 3. The fully enclosed containment system can
also be divided into cold containment and hot containment types.
Fig. 3. The configuration of the aisle containment.
5
In a freely open cold-aisle data center, hot air infiltrates into the cold
aisle from the top and sides, which increases the rack inlet temperature.
Capping of the cold and the hot aisle is an effective way to improve hot
spots and control the inlet temperature of IT equipment (Alkharabsheh
et al., 2014e; Ghosh et al., 2012; Nada et al., 2016b; Wang et al., 2017).
Using the isolation technique with aisle partitioning or aisle contain-
ment, cold air does not mix with the hot air; hence, the rack inlet tem-
perature can be reduced by ~3–13% for aisle partitioning and by
~13–40% for aisle containment at different power densities (Gondipalli
et al., 2008; Nada et al., 2016a). Additionally, the percentage of the en-
hancement of data center thermal management due to using cold aisle
containment increases with increase of the rack power density (Nada
and Elfeky, 2016). Therefore, a containment system also reduces the
overall cooling energy cost (Arghode et al., 2013b; Shrivastava et al.,
2012). Because air containment reduces unnecessary loss of air flow
supplied by the CRAC, it can bring energy savings of fan power, approx-
imately 15% of the air conditioner power demand (Takahashi et al.,
2008). In addition, based on this cooling efficiency, the supply air tem-
perature can be increased from 18 °C to 22 °C by using aisle containment
technology (Cho et al., 2014). Then, high chill-water temperature can
satisfy the load demand, resulting in additional energy savings of
8.9 kW. Thus, 59% of the energy required for the CRACs used in a tradi-
tional open type data center can be saved (Schmidt et al., 2012). Al-
though the aisle containment prevents or reduces the mixing of cold
and hot air streams, air can also escape from the crack of the door.
There is a certain threshold value of leakage beyond which the contain-
ment does not have benefits over uncontained systems. This value has
been found to be about 15% of the total containment surface area
(Alkharabsheh et al., 2014f). It is also interesting that soft containment
systems show relatively lower loss in cooling capacity than hard con-
tainment systems (Khankari, 2014).
2.1.4. Perforated tiles
Perforated tiles offer substantial resistance to airflow from the ple-
num, and the flow through the perforated tiles is influenced by several
factors, such as open area, pore shape, pore size, tile width, edge block-
age, and airflow outlet angle.
2.1.4.1. Open area. Whether the air can flow from the raised floor ple-
num to the cold aisle depends mainly on the motive power provided
by CRAC and the resistance of the perforated tiles. The open area is
the main factor affecting the resistance of the perforated tiles. A
large open area makes it easy for the air to flow from the raised
floor plenum to the cold aisle, and reverse airflow occurs simulta-
neously. For a fixed layout and plenum height, most studies
(Bhopte et al., 2006b; Karki et al., 2003; Nagarathinam et al., 2013;
Schmidt et al., 2001; VanGilder and Schmidt, 2005; Wang et al.,
2015) have indicated that the uniformity of flow rate is related to
the open area of the perforated tiles. At low open area values, the
flow rates of all perforated tiles are almost equal due to the uniform
pressure beneath the floor. Additionally, the reverse flow near the
CRAC is eliminated when the open area is reduced to 15% (Schmidt
et al., 2001). However, the uniform distribution caused by reducing
the open area may result in the loss of chilled air from the openings
around cables and pipes or the gaps between tiles. Then, this air can-
not be used by the IT equipment. At the same time, the air flow rates
supplied by the CRACs are also reduced, because the CRAC unit
blowers must overcome the total resistance, which increases with
decrease of the open area of the perforated tiles. A 25% opening not
only showed much better perforated tile flow uniformity alongside
the tiles (VanGilder and Schmidt, 2005; Wang et al., 2015) but also
presented the lowest maximum temperature (Nagarathinam et al.,
2013).
2.1.4.2. Tile geometry. The tile geometry also has a significant influence
on the airflow characteristics above the tile. Perforated tiles can have
6 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
pores of various shapes with curved edges. A very intriguing finding is
that tiles with non-symmetrical geometrical patterns result in non-
symmetrical exit velocity profiles (Abdelmaksoud et al., 2010). How-
ever, the flow fields through a circular and square pore are similar
(Arghode and Joshi, 2013). The pore size of tiles has a small but non-
negligible effect on the flow field, especially near the tile surface at
both the rack and aisle ends. This occurs because, assuming that the
open area is constant, smaller pore size increases the number of pores,
which results in a smaller pore-to-pore center spacing and faster
decay of the closely spaced jets. Thus, the effect region of entrainment
and flow development above the tile surface are reduced. This is in
agreement with the results of Ref. Abdelmaksoud et al. (2010). Another
key factor influencing the flow through the tiles is the equivalent diam-
eter ratio, which equals the ratio of the thickness of the hole to the di-
ameter of the hole. With the same value of porosity and equivalent
diameter ratio, the pressure difference between the inlet and outlet is
the same (Ling et al., 2017). In addition, if the air that can be delivered
to the top end of the rack is related to blocked tile edges, higher tile
edge blockage may be detrimental for a fully provisioned case but help-
ful for an under-provisioned case. In particular, lower effective porosity
with blocked edges results in higher air momentum above the tile sur-
face and hence more cold air reaches the top (Arghode and Joshi,
2015; Arghode and Joshi, 2016).
2.1.4.3. Tile orientation and outlet angle. Tile orientation and outlet angle
influence the air outlet direction and also affect the uniformity of the
cold air supply. Seymour (2015) investigated three tile orientations:
parallel with the flow, at 45° to the cross-flow direction beneath the
tile, and perpendicular to the cross-flow direction beneath the tile. The
results of the three tile orientations demonstrated sensitivity in bulk
flow rate and considerable variations in distribution of flow (Seymour,
2015). Additionally, the airflow outlet angle (30°, 45°, 60° and 90°) af-
fects the airflow distribution, and it was found that the air distribution
is better at the angle of 60° than at other angles (Ni et al., 2017).
2.1.4.4. Air leakage. As discussed in Section 2.1.2, the chilled air can be
lost through gaps in the perimeter walls and gaps between the panels
on the raised floor (Karki, 2007). The leakage flow in a conventional
data center is around 5–15% of the available cooling air (Alkharabsheh
et al., 2014c). In other words, if there are 10 CRACs in a data center
with a leakage area of 10%, one unit can supply the leakage flow. Reduc-
ing the air leakage in a data center can reduce the energy consumption
of CRACs; however, the inlet air temperature of the cabinet will be in-
creased simultaneously. The leakage can result in a decrease in the aver-
age rack inlet temperature of 0.7–0.9 °C for every 10% reallocation of
leakage flow to a desired rack inlet location (Hamann et al., 2008).
With increasing awareness toward reducing ineffective airflow and im-
proving the energy efficiency of CRACs, in new data centers, the leakage
area is lower than it is in old data centers and the leakage ratio could be
within 0.1–0.2% (Radmehr et al., 2005).
2.2. Rack level
2.2.1. Porosity ratio of rack door
Rack doors contribute to the increase of airflow resistance, so
these should be eliminated if possible. Even so, the outlet flow of
the rack is usually obstructed by power and network cables. Thus,
additional fans or blowers need to be installed in the rack to over-
come the resistance. The effect of different porosity ratios of front
and rear door on cooling effect has been investigated using a numer-
ical method. By increasing the porosity ratio from 25% to 50%, the
temperature of the airflow in the rack is reduced and the thermal en-
vironment inside the rack is improved significantly. This occurs be-
cause the heat generated by the IT equipment can be efficiently
discharged with the higher porosity ratio (Ling et al., 2015; Zhang
et al., 2016). If doors are absolutely needed, the result suggests a
minimum open area of 60% to allow for maximum airflow (http://
www.thefreelibrary.com/Effective+computer+room+cooling.+%
28Infrastructure%29.-a0106646787).
2.2.2. Airflow rate and inlet air temperature
Air flow rate and air temperature are key parameters describing the
thermal environment in a data center. Reduced air flow rates with the
same heat loads always result in higher air temperature differences
across the rack and thereby higher temperature in the rack (Schmidt
and Cruz, 2003). On the contrary, increased airflow can increase the
heat transfer rate and decrease the local temperature of the chassis or
rack as a result of the fully turbulent flow regime (Arghode and Joshi,
2015; Ni et al., 2017). For these reasons, the airflow rate and inlet air
temperature need to be considered together. To ensure that the rack
inlet air temperature is b27 °C as recommended by ASHRAE, the airflow
for a 1 kW load must be N52 l/s (Priyadumkol and Kittichaikarn, 2013).
In addition, with the recommend temperature range of ASHRAE
expanding and awareness of energy saving increasing, high-
temperature air is often supplied to the servers. Although, the practice
can save energy and decrease the load on the cooling system due to
the application of the free cooling, the server power increases for air
inlet temperature increases from 24 to 27 °C and from 27 to 31 °C is in
the range of 4–7% and 11–16%, respectively (Iyengar et al., 2012). There-
fore, optimizations must be carried out to address the relationship be-
tween energy consumption reduction of CRACs and power increase of
servers.
2.2.3. Server population, arrangement, and power density
With the rapid development of the IT industry, the power density
of the rack has also increased as a result of the server population, ar-
rangement, and server types. The server population has a significant
impact on the rack-level air temperature. As the number increases,
the exhaust temperature of the rack increases (Ghosh et al., 2012).
In addition, the servers' power varies according to the IT computing
loads. Servers in low or high states of utilization have different ef-
fects on the room power density and air flow distribution. High-
power-density servers not only dissipate more heat into the room
but also require more air to cool the internal components of the
servers to within allowable temperatures (Alkharabsheh et al.,
2014d). More importantly, the low-power condition and high-
power condition are converted from one to the other according to
the users' application. Therefore, it is difficult to create a suitable
thermal environment, particularly for low-power to high-power
conversion. Keeping the lower-supply air could alleviate this difficult
situation, but it requires more energy consumption.
Server arrangements include uniform distribution, discrete distribu-
tion, segmented distribution, and clustered distribution (Fakhim et al.,
2013; Kumar et al., 2011; Zhu et al., 2015). In general, it is easy for
servers in discrete distribution or segmented distribution to suck cold
air because no or few other servers compete for the air. Especially, for
high-powered servers, it is not suitable to cluster the servers together
in the rack because their mutual competition results in the need for a
large amount of recirculation (Rambo and Joshi, 2005). In addition,
hotpots can appear easily in these regions. If a clustered arrangement
is required, keeping a server at the highest possible location in a rack
is a more energy-efficient practice than placing it at a middle or bottom
location (Ghosh et al., 2012). If the servers are uniformly installed in a
rack, temperature spikes and high average temperatures may occur at
the high-position regions. This may occur because the low-position
servers over-consume the cooled air resulting in an air supply reduction
and hot air recirculation along the rack height (Nada and Elfeky, 2017).
Relatively speaking, uniform installation is the best choice, except for
the high-density rack. However, clustering in the upper rack could be
an acceptable substitution.
 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
2.3. Server level
The server fans provide the power to transfer air into the servers and
prevent the internal electronic components, e.g., CPU, memory, and
disk, from overheating. A safety precaution is to increase the server
fan speed to increase the air flow rate and provide better cooling within
the servers. However, the fan may run at an unnecessarily high speed
when the server is idle, and increasing the speed requires an exponen-
tial increase in fan power (Garraghan et al., 2016). Ideally, an increase in
fan speed across the server at a fixed power level decreases the temper-
ature difference between the inlet and the outlet (Alkharabsheh et al.,
2014d) because increasing the airflow rate increases the forced convec-
tion effect, and then the heat-transfer coefficient values (Ibrahim et al.,
2012b). In other words, with increase in the speed of the server cooling
fan, the server thermal resistance shows a decreasing tendency (Ham
et al., 2015). This result is agreement with Ref. Iyengar et al. (2012),
the thermal resistance for a 7300 rpm fan speed is close to 0.3 °C/W
and that for a 13,000 rpm speed is approximately 0.2 °C/W. In addition,
the minimal fan speed has a near-linear relationship with the inlet tem-
perature (Chen et al., 2015). More importantly, this relationship varies
with CPU utilization and CPU temperature. Several studies have investi-
gated the relationship between fan speed and temperature decrease.
The temperature difference is noticeable for the Random Access Mem-
ory (RAM), for which the temperature decreases about 10 °C as the
flow rate is increased to 21 CFM (cubic feet per minute). In addition, a
change in flow rate of 6.8% of the free delivery flow rate can cause an ap-
proximately 10 °C change in temperature at certain locations
(Alkharabsheh et al., 2014a).
Moreover, the server fin is an effective strategy for causing the upris-
ing cold air to be sucked by the servers. Server-fin designs, combinations
of shapes, and opening angles, are evaluated by transient analysis, vor-
ticity analysis, and temperature change rate analysis (Sakaino, 2014).
Curved server fins have yielded higher vorticities around the compo-
nents than flat server fins, and this difference can be assumed to attract
uprising cool air from the perforated floor better, leading to more effi-
cient heat transfer. The average performance of accelerating cooling
was enhanced by a factor of 2 by server fins compared with no fin
server. The difference in temperature increment rate between the
cases of server fins and no fin cases is small.
3. Numerical study
3.1. Simulation software
Due to the complex nature of flow inside the data center, computa-
tional fluid dynamics (CFD) is an efficient solution for investigating the
airflow distribution within the data center. With the development of
CFD, easy to use, accurate, and inexpensive computational design and
analysis tools have overcome the complex and sometimes conflicting
thermal requirements. Simulation software can enhance features and
functionality. Through simulation and analysis, users can reduce prod-
uct design costs, operational and maintenance costs for data center in-
frastructure, and thereby PUE values, increase data center availability,
and extend data center life. In China, the Code for Design of Data Centers
(GB 50174-2017) and the Rating Standard for Data Centers (T/CECS 488-
2017) suggest that the airflow distribution should be simulated and
predicted using CFD methods. The most popular simulation tools ap-
plied in the data center are ANSYS Fluent, ANSYS CFX, ANSYS Icepak,
FloTHERM, Airpak, PHOENICS, FloVENT, TilFlow, and 6 SigmaDCX.
These tools can be divided into general and professional software.
The general software packages, such as ANSYS Fluent, ANSYS CFX,
Airpak, PHOENICS, FloVENT, are not specifically used for data centers.
They can also be employed to analyze other flow fields. ANSYS Fluent
and ANSYS CFX are the most-powerful CFD tools for flow and heat
transfer analysis, and these are employed to investigate the perforated
tile and the flow distribution in a data center (Abdelmaksoud et al.,
7
2012; Arghode and Joshi, 2013; Arghode et al., 2013a; Chen et al.,
2006; Li et al., 2010; Ling et al., 2017; Nada et al., 2016b; Priyadumkol
and Kittichaikarn, 2013; Singh et al., 2011; Siriwardana et al., 2012;
Wang et al., 2011). Airpak and PHOENICS are accurate, quick, and
easy-to-use design tools that simplify the application of state-of-the-
art airflow modeling technology to the design and analysis of ventila-
tion systems, which are required to deliver indoor air quality, thermal
comfort, health and safety, air conditioning, and/or contamination con-
trol. These are usually applied in airflow or cooling effect simulation of
the data center (Ling et al., 2015; Stavreva et al., 2016; Yuan et al.,
2018; Zhang et al., 2014b; Zhu et al., 2015). FloVENT is designed specif-
ically for engineers involved in the design and optimization of heating,
ventilation, and air conditioning (HVAC) systems (https://www.
mentor.com/products/mechanical/flovent/). Accordingly, FloVENT is
mostly used for data center thermal performance analysis and cooling
optimization (Fakhim et al., 2013; Nagarathinam et al., 2013; Pardey
et al., 2015; Patel et al., 2001; Romadhon et al., 2009; Srinarayana
et al., 2014; Zhang et al., 2008).
The professional software can be classified into part-level and full-
level tools. ANSYS Icepak and FloTHERM are representative simulation
tools for the electronic level. ANSYS Icepak is widely used to solve
component-level (chip level), board-level (printed circuit board level),
and system-level (server level) thermal problems. FloTHERM is used
to predict airflow, temperature, and heat transfer in components,
boards, and complete systems, including racks and data centers
(Alkharabsheh et al., 2014a; Alkharabsheh et al., 2014b; Bedekar,
2006; Bhopte et al., 2006d; Gondipalli et al., 2008; Ibrahim et al.,
2012a; Iyengar et al., 2005; Iyengar et al., 2007; Rodgers and Eveloy,
2003; Shrivastava, 2008; Song, 2016b; Song et al., 2014a, 2014b). It is
the undisputed world leader for electronics thermal analysis (https://
www.mentor.com/products/mechanical/flotherm/). In general, Icepak
and FloTHERM can provide almost the same level of performance
(Yang, 2000). However, in the majority of experiments, Icepak com-
puted junction and case temperatures have been slightly more accurate
than those calculated by FloTHERM and the run time has also been
lower (Zahn, 1998). TileFlow is a powerful three-dimensional CFD
modeling software tool for room-level cooling performance analysis of
data centers (http://tileflow.com/tileflow/data-center-cfd-modeling-
software-overview.html). It is usually applicable to both raised-floor
and non-raised-floor data centers (Hamann et al., 2008; Karki et al.,
2003; Radmehr et al., 2005; Radmehr et al., 2013; Schmidt and Cruz,
2004; Vallury et al., 2015). Recently, 6 SigmaDCX has been committed
to data center design, management, and thermal analysis (Alissa et al.,
2015a; Alissa et al., 2015b; Nemati et al., 2016; Shrivastava et al.,
2012). It solves the full level of thermal issues and includes room level
6 SigmaRoom/RoomLite (thermal analysis for the room, environment,
etc.), rack level 6 SigmaRack (system-level thermal analysis for chassis
cabinets and in-bay components), and electronic level 6 SimgaET (ther-
mal analysis for electronic modules, PCBs, chip packages, etc.) (https://
www.futurefacilities.com/). Summaries of the applications of the main
numerical tools and their characteristics are provided in Tables 1 and
2, respectively. It can be seen that TileFlow and 6 SigmaDCX have low
difficulty for software users. TileFlow is good at simplifying physics
and meshing scheme (not a 3D mesh under the floor and a coarse 6-
inch mesh above floor), but this cannot capture the true behavior of
the flow. In comparison, 6sigmaDCX can not only achieve the accurate
simulation results, but also allow the user to enter details users know
are important to the results, and modeling of many control systems.
Other simulation tools such as STAR CCM+ and ThermoStat are also
employed in data center thermal performance analysis. STAR CCM+ is
used to predict the server outlet temperature through the flow rate
(Fulpagare and Bhargav, 2015) and server heat source excitation in an
actual data center and analyze the transient characteristics of the rack-
level thermal environment (Fulpagare et al., 2018). ThermoStat is a
server and rack-level thermal modeling tool, which has been applied
in the thermal profiling of rack-mounted servers (Choi et al., 2008).
8 C. Jin et al. / Science of the Total Environment 695 (2019) 133801

Table 1
The application of the common simulation tools.

Software Reference The aim of research Turbulence model

ANSYS Abdelmaksoud Discuss about modeling techniques of the perforated tile Two-equation realizable k-ε turbulence
Fluent et al. (2012) model coupled with standard wall
functions
Ling et al. (2017) Perform the flow distribution of perforated tiles for different pore-type and flow parameters Realizable k-e model
Arghode and Joshi Used model for air flow through the perforated tiles, the porous jump model, specifies a step Standard k–ε model
(2013) pressure loss across the tile surface without affecting the velocity field across the tile
Nada et al. (2016b) Investigate data centers' thermal performance for different configurations of CRACs units and k–ε model
aisles separation
Priyadumkol and Study the air flow through perforated tile for air conditioning system in data center Standard k-ε turbulence model
Kittichaikarn
(2013)
Li et al. (2010) Find out the most effective and low-cost air condition system RNG k-ε turbulence model
Wang et al. (2011) Take various factors, such as cooling systems and server layout, as inputs to analyze the Standard k-epsilon two-equation
thermal conditions of the data center turbulence model
Arghode et al. Report the effect of including more resolved tile geometrical details on the solution accuracy Realizable k-e model
(2013a) at the expense of increased computational effort
ANSYS CFX Chen et al. (2006) Study the flow domain of the under-floor plenum in a raised-floor data center based on the
analysis and flow control strategies
Singh et al. (2011) Apply CFD in a data center to improve its operational efficiency RANS model
Siriwardana et al. Upgrade the computing equipment with minimal thermal impact on the existing optimized Shear Stress Transport turbulence model
(2012) data center cooling environment
Airpak Ling et al. (2015) Analyze the way of improving the cooling effect of the data center with the UFAD system Interior zero-equation model
Zhang et al. Investigate a solar chimney integrated with under-floor distribution system in data center The indoor zero-equation model
(2014b)
Yuan et al. (2018) Investigate the effect of flexible baffles on airflow distribution in high-density data center Standard k-ε turbulence model with RANS
model
PHOENICS Zhu et al. (2015) Investigate the airflow distribution, as well as the cooling effects of different servers' layout Standard k–ε turbulence model
Stavreva et al. Simulate of a data center by using analysis the most appropriate size and numerical network Standard k–ε turbulence model
(2016) model of turbulence were selected
FloVENT Romadhon et al. Focus on the optimization cooling system in data center by arrangement of racks position and
(2009) airflow management
Pardey et al. (2015) Benchmark the level of agreement that can be practically obtained and also to investigate the LVEL k–ε turbulence model
level of modeling detail required
Nagarathinam et al. Optimize the layout of a raised-floor data center with respect to rack and CRAC position, Standard k–ε turbulence model
(2013) raised-floor height, ceiling height, and percentage opening of perforated tiles
Srinarayana et al. Investigate the thermal performance of an air-cooled data center with raised-floor and Standard k–ε turbulence model
(2014) non-raised-floor configurations
Zhang et al. (2008) Investigate the effect of server rack model detail on the prediction of temperature distribution k–ε with Stratification, LVEL Algebraic,
within the data center test cell Capped LVEL model
Patel et al. (2001) Study the inlet air temperature to the system and find the hot spots in the room k–ε turbulence model
Fakhim et al. Investigate the thermal performance of semi-populated racks with different server Standard k-ε turbulence model
(2013) arrangements
ANSYS Mulay (2010) Analyze data center cooling strategies and the impact of the dynamic thermal management on Standard k-ε turbulence model
Icepak its energy efficiency
Tan et al. (2007) Focus on modeling the server-rack air flow and heat transfer interaction in a data center Two-equation standard k-ε turbulence
model
FloTHERM Alkharabsheh et al. Investigate the effect of server, rack, and CRAH heat capacity in cases of server power k-ε turbulence model
(2014b) fluctuations and CRAH failure situations
Bhopte et al. Discuss the effect of under floor blockages on data center performance
(2006d)
Iyengar et al. Understand the inherent error in the model prediction of flow and temperature distributions Standard k-ε turbulence model
(2007) in data center
Bedekar (2006) Investigate the effect of CRAC location on a fixed rack layout of a data center k-ε turbulence model
Iyengar et al. Study the thermal characterization of non-raised floor air cooled data centers k-ε turbulence model
(2005)
Ibrahim et al. Introduce transient modeling of data centers with changing power dissipations and CRAC Standard k-ε turbulence model
(2012a) airflow rates
Shrivastava (2008) Compare the thermal performance of seven different data center airflow duct designs k-ε turbulence model
Song et al. (2014b) Analyze multiple hot aisle/cold aisle data center configuration
Song et al. (2014a) Verify the zonal model based on the velocity propagation method
Song (2016b) Analyze the compact model of fan-assisted tile imported into a basic hot aisle/cold aisle data Standard k-ε turbulence model
center
Rodgers and Eveloy Systematically assess the numerical predictive accuracy for board-mounted electronic k-ε turbulence model
(2003) component heat transfer
Alkharabsheh et al. Investigate the influence of the rack structure on the thermal field Standard k-ε turbulence model with RANS
(2014a) model
Gondipalli et al. Study the effect of isolating cold aisle on rack inlet temperature k-ε turbulence model
(2008)
TileFlow Karki et al. (2003) Discuss a number of techniques that can be used for controlling airflow distribution Standard k-ε turbulence model
Radmehr et al. Combine measurement and simulation to help the uses of CFD technology to create an
(2013) accurate model of raised floor data center
Radmehr and Outline a procedure to measure leakage flow, provide data on the amount of the distributed Standard k-ε turbulence model
Schmidt et al. leakage flow, and to show the quantitative relationship between the leakage flow and the
(2005) leakage area
 C. Jin et al. / Science of the Total Environment 695 (2019) 133801 9

Table 1 (continued)

Software Reference The aim of research Turbulence model

Schmidt and Cruz Investigate the effect on inlet rack air temperatures as a result of mal-distribution of airflows k-ε turbulence model
(2004) exiting the perforated tiles located adjacent to the fronts of the racks
Vallury et al. Demonstrate the challenges associated with a standard air cooled approach in a high
(2015) performance computing data center
Hamann et al. Investigate the impact of air flow leakage on server inlet air temperature in a raised floor data k-ε turbulence model
(2008) centers
6 SigmaDCX Shrivastava et al. Compare the performance of standard Hot Aisle/Cold Aisle arrangement vs. Containment k-ε turbulence model
(2012) System
Alissa et al. (2015b)
Investigate the rapid modeling technique relied on experimentally measured (for IT) or
approximated (for CRAH) flow curves
Nemati et al. Evaluate the interaction of perimeter and row-based cooling for cold-aisle containment Standard k-ε turbulence model
(2016) environment
Alissa et al. (2015a) Illustrate basic guidelines for integrating empirical measurements to state of the art modeling
approaches
STAR CCM+ Fulpagare and Predict the server outlet temperature through the flow rate and server heat source excitation
Bhargav (2015) in an actual data center
Fulpagare et al. Transient analysis to characterize the rack level thermal environment
(2018)
ThermoStat Choi et al. (2008) Introduce the tool for studying temperature in rack-mounted servers

3.2. Numerical simulation and validation
With the development of simulation technology and considering the
expense in the experiments, simulation tools are now widely employed
in the thermal environment prediction of data centers. However, the
data center simulation is a multi-scale problem, the magnitude of
which is from nm to m. This will seriously affect the mesh generation
and computing speed of the model. Besides, tens of thousands of the
servers are in a data room, and their brands and heat dissipation are
not identical, which will create significant barriers to device statistics
and simulation. Hence, simplification of the model is necessary and
has become progressively more acceptable in simulation, especially
for IT equipment. The accuracy of the simplified simulation results is
associated with the following aspects: (1) the model, (2) mesh genera-
tion, (3) boundary conditions, (4) turbulence model, and (5) experi-
mental validation.
3.2.1. The model
Like any simulation, the model should be built based on the actual
data centers, including the main geometries of the data room, IT equip-
ment, CRACs, etc. In the simulation, the architecture of the envelope is
normally regarded as the shape of the room. This is because the internal
heat gains are so large that the fabric heat transfer could be ignored, es-
pecially for the rooms without outside windows. As for the internal
parts, the users should pay particular attention to the server arrange-
ment, and leakage airflow, which play an important role in airflow
path. Hence, for the simplification of the simulation, it can be a good
choice that using a black box approach for the server geometry, the pa-
rameters placed at the vents are imposed in the rack and room models
for accurate flow and heat distribution (King et al., 2014; Zhai et al.,
2012). Additionally, the main obstructions under the floor and the leak-
age areas between tiles should be considered in the simulation, includ-
ing the location, size and penetrations in the raised floor beneath the
equipment.
3.2.2. Mesh generation
Historically, one of the most time-consuming issues for a CFD mod-
eler has been defining the solution grid or mesh. The choice and level of
refinement can substantially impact the predicted airflow and heat
transfer or even the ability for a converged solution to be achieved.
The simplest form of grid is one where the space is divided up into an
array of rectangular elements. It presents a set of parallel lines across
the page and a similar set running vertically up the page. However, in
the solution domain, a small gap between lines in one direction with
large gaps in another will result owning to the high aspect ratio and
high expansion ration. In three dimensions, the structured Cartesian
grid is widely used, which has six faces and next to each cell face is an-
other cell with a coincident face. Although some CFD tools prefer un-
structured grid to more easily describe complex shapes and more
accurately predict surface effects, it is time-consuming in data center
simulation. Hence, the simplest way is to retain the Cartesian approach,
but to no longer insist on the lines extending all the way through the so-
lution domain, and to allow a cell face to have more than one other cell
faces next to it. In addition, an alternative approach is not to insist on the
cell shape being rectangular or box shaped. This allows the grid to fol-
low more complex outlines and permits the cell faces to more closely re-
flect the true surface (Geng, 2015).
3.2.3. Boundary conditions
In a whole data center simulation, it is a challenge to compute a nu-
merical model considering all of the length scales. The ideal methodol-
ogy is that using a compact model in the package-level design, the
servers can be replaced by an equivalent thermal and flow network in
the rack-level modeling. Then, in the room-level model, the entire
rack can be replaced by a segmented flow model which allows for var-
iations of pressure and temperature from top to bottom. According to
the coupling between the various levels and the supply and return air
conditions, iteration from one level to the next and back again is re-
quired until a stable profile at the rack inlet and outlet is obtained
(Tan et al., 2007). However, it is complicate and time-consuming. In
general, the main input parameters are the power, airflow rate, and
air temperature. As for the IT equipment that is replaced by black box,
the main boundary conditions are the equipment power and airflow
rate, instead of heat sources (i.e., the chips) and server fans. The
power value is usually adopted from the monitoring system, and the
nominal airflow per kW of IT load typically of the order of
56.67 l/s/kW of IT load. If the power of IT equipment is known, and
the air temperature difference between inlet and outlet vents of IT
equipment is obtained from equipment manufacturers or field experi-
ments, then, the airflow rate can also be calculated according to the for-
mula Q = cmΔt. In particular, if only the rack power is available, the
power of IT equipment is distributed proportionally by the rack power
according to the nameplate power weight. As for CRACs, of greater im-
portance is the representation of cooling capacity and the controls for
air volume, cooling, and any humidification/dehumidification, if appli-
cable. Particularly, the location of air temperature sensor is dramatically
important for air supply control or return air control.
3.2.4. Turbulence model
Turbulence model is a key issue in most CFD simulations, and is nec-
essary because virtually all engineering applications are turbulent. In
10 C. Jin et al. / Science of the Total Environment 695 (2019) 133801

Table 2 Therefore, to improve thermal performance and minimize the energy

The characteristics of main simulation tools. cost of a data center, it is necessary to analyze its thermal behavior. Cur-
Software Major in Difficulty rently, there are various metrics for assessing the airflow performance
Level for of a data center.
data center As mentioned above, in the process of the air transfer, it is inevitable
simulation
that hot air recirculates to the supply air or that the cool air bypasses to
ANSYS None High the return air. To evaluate the degree of mixing of the airflow, the supply
Fluent
heat index (SHI) and return heat index (RHI) were proposed by Sharma
ANSYS CFX None High
Airpak HVAC system Moderate et al. (2002). Considering the law of conservation of mass, SHI + RHI =
PHOENICS HVAC system Moderate 1. For the ideal air distribution, the value of SHI is equal to 0 and the
FloVENT HVAC system Moderate value of RHI is equal to 1. Usually, a value of SHI b 0.2 or a value of RHI
ANSYS Electronic equipment Moderate N 0.8 can be regarded as a good airflow distribution. The β can capture
Icepak
FloTHERM Electronic equipment Moderate
the local inefficiency, and the value of β is different for different racks.
TileFlow Cooling performance for data center Low Some parts of the rack may show a β value of 1, or possibly a value N1,
6 SigmaDCX Cooling and power performance for electronic Low but the averaged value is always between 0 and 1. A β value of 0 indi-
equipment, rack, room in data centers cates no impact of hot-air recirculation at that location (Schmidt et al.,
2005). Although some of the above evaluation indexes can describe
some characteristics of the data center airflow distribution, the calcula-
data centers, according to the different research objects, the turbulence tions of all these indexes are based on the rack inlet air temperature and
models can also be chosen differently. As depicted in Table 1, the stan- exhaust air temperature. Because it is easy to cover the temperature
dard k-ε turbulence model with standard wall functions can satisfy level of the airflow, a lower weighted average temperature of some cab-
the general simulation calculation, and the results could match the mea- inet may result. In fact, there are some high-temperature return airflows
surement results. However, if the study is about a part of the data center, in parts of the cabinet. Considering the above, the index of mixing (IOM)
such as perforated tiles and racks, the realizable k-ε turbulence model is has been proposed to evaluate and describe the degree of penetration
good for only tile models, whereas, the body force and modified body and mixing of airflow in the cabinet. It can characterize the distribution
force models are the best choices for the tile model at the server rack of the thermal environment in the data center well and facilitate finding
level (Phan et al., 2019). the locations of potential hotspots and making corresponding improve-
ments. A higher value of IOM indicates a greater the possibility of a local
3.2.5. Experimental validation hot spot at the location of the rack. The heat weighted index of mixing
In the validation stage, the air temperature of the racks, perforated (HIOM) is an improved based on the IOM that can be used to compare
tiles and CRACs, and the airflow of the perforated tiles and CRACs, are the overall or local thermal environment among different data centers
the main parameters regarded by modelers as objectives for evaluating visually. The HIOM characterizes the thermal health of the data center
the simulation results. Generally, the root mean square error is widely and also reflects the airflow behavior accurately. It can be regarded as
used as the accuracy criterion of the simulation results. If the value is a comprehensive evaluation index for the thermal environment in the
b15%, the simulation value can be considered to be in good agreement data center. A higher value of HIOM indicates lower thermal reliability
with the onsite measurement. If not, the simulation should be refined. (Tian et al., 2012). Shu (2014) claims that two racks can have the
The possible sources of inaccuracies in the model are cable densities, same index value of IOM and the same local hot spots but that their
IT flow rates, rack or containment leakage, model convergence, model rack inlet temperatures are partly different (see Fig. 4). Hence, the
grid sensitivity, IT load distribution, and CRACs details. Among them, heat mixing index (HMI) and heat weighted heat mixing index
the primary objects of refinement are IT flow rates, IT load distribution (HHMI) were proposed based on the IOM to characterize the rack ther-
and CRACs details. mal environment. Herrlin (2005, 2008) proposed the rack cooling index
(RCI) and return temperature index (RTI) in 2005 and 2007, respec-
4. Airflow performance metrics and thermal optimization tively. These metrics directly address whether the CRAC can efficiently
cool the IT equipment. There are two RCIs: RCILo measures overcooling
4.1. Performance metrics of the rack, inlet temperature below the lower limit of recommendation,
and RCIHi tests undercooling of the rack, inlet temperature above the
In recent years, due to high-performance IT equipment, the airflow higher limit of recommendation. RCILo or RCIHi values closer to 100% in-
distribution is directly linked to reliability and energy savings. dicate better airflow distribution. The RTI captures the relative degree of

Fig. 4. Assumption of the rack air distribution (Shu, 2014).

 C. Jin et al. / Science of the Total Environment 695 (2019) 133801 11

Table 3
Summary of most thermal metrics and benchmark values.

Introduced by Metric Explanation Information Input parameters Formula Applied Benchmark value
provided level

Sharma et al. SHI Supply Heat Quantify the Airflow supply ∑i ∑ j ðT rini; j −T Csup Þ Rack Ideal 0
(2002) Index extent of temperature from CRAC SHI ¼ level Good b0.2
∑i ∑ j ðT rout i; j −T Csup Þ
mixing of unit, rack inlet and outlet Room
return air airflow temperatures level
Sharma et al. RHI Return Heat Quantify the Mass flow rate supply RHI = 1 − SHI Rack Ideal 1
(2002) Index extent of from CRAC unit, mass level Good N0.8
bypass of flow rate across Room
supply air equipment, airflow return level
temperature at CRAC, air
flow supply temperature
from CRAC unit, airflow
temperature exhaust
from rack
Schmidt et al. β Beta Index Presence of Local airflow inlet supply T rin ðzÞ−T Csup Rack Ideal 0
(2005) Index recirculation and outlet temperature β¼ r level
T out −T rin
and over
heating
Tian et al. IOM Index of Mixing degree The maximum rack inlet T i; max −T i; min Rack Poor N1
IOM ¼
(2012) Mixing of airflow in air temperature, the T o −T i level Close to
rack level minimum rack inlet air 1
temperature, the mean
rack inlet and outlet air
temperature
Tian et al. HIOM Heat Mixing degree IT heat dissipation, the X
N
Qi Room Poor N1
(2012) Weighted of airflow in total IT heat dissipation HIOM ¼ IOM i level Close to
i
QN
Index of rack level 1
Mixing considering
the difference
of heat
dissipation
Shu (2014) HMI Heat Mixing Thermal The maximum rack inlet T i; max þ T i −ðT ref þ T i; min Þ Rack
HMI ¼
Index environment air temperature, the T o −T i level
within the rack minimum rack inlet air
temperature, the mean
rack inlet and outlet air
temperature, outlet air
temperature from CRAC
Shu (2014) HHMI Heat Thermal IT heat dissipation, the X
N
Qi Room
Weighted environment total IT heat dissipation HHMI ¼ HMI i level
i
QN
Heat Mixing within the rack
Index considering
the difference
of heat
dissipation
P
Herrlin RCILO Rack Cooling Quantify the Rack intake air ðT min−rec −T x ÞT x bT min−rec Rack Ideal 1
RCILO ¼ ½1−  n  100%
(2005, Index Low rack health at temperatures ðT min−rec −T min−all Þ level Good ≥96%
2008) the low end of distribution, maximum Acceptable 91–95%
the recommended and Poor ≤90%
temperature allowable temperature
range according to some
guideline or standard
P
Herrlin RCIHI Rack Cooling Quantify the Rack intake air ðT x −T max−rec ÞT x NT max−rec Rack Ideal 1
RCIHI ¼ ½1−  n  100%
(2005, Index High rack health at temperatures ðT max−all −T max−rec Þ level Good ≥96%
2008) the high end of distribution, minimum Acceptable 91–95%
the recommended and Poor ≤90%
temperature allowable temperature
range according to some
guideline or standard
Herrlin RTI Return Presence of Server inlet and outlet air T Cret −T Csup Rack Ideal 100%
(2005, Temperature recirculation or temperature, return air RTI ¼  100% level Good 95–105%
T rout −T rin
2008) Index bypass temperature to CRAC,
phenomena discharge air temperature
from CRAC
VanGilder CI Capture The fraction of Mass of species and total mass of species Rack Ideal 1
CI ¼
and Index air ingested by mass of fluid total mass of fluid level
Shrivastava the rack that
(2006, originates from
2007) local cooling
resources
VanGilder RI Recirculation The percentage Mass of recirculation and RI = 1-CI Rack Ideal 0
and Index of airflow total mass of fluid level
Shrivastava ingested by a
(2006, rack that

(continued on next page)

12 C. Jin et al. / Science of the Total Environment 695 (2019) 133801

Table 3 (continued)

Introduced by Metric Explanation Information Input parameters Formula Applied Benchmark value
provided level

2007) originates
outside the
cold aisle
Tozer et al. NP Negative Airflow Return air temperature to T uf C Room Ideal 0
sup −T sup
(2009) pressure infiltration into CRAC, discharge air NP ¼ level
T Cret −T uf
sup
ratio underfloor temperature from CRAC,
plenum floor void temperature
Tozer et al. BP Bypass ratio The rate of air Return air temperature to T Sout −T Cret Room Ideal 0
(2009) which leaves CRAC, floor void BP ¼ level Good 0–0.05
T Sout −T uf
sup
the floor grills temperature, server Acceptable 0.05–0.2
and returns outlet air temperature
directly to the
CRAC unit
without
cooling servers
Tozer et al. R Recirculation The rate of hot Server inlet and outlet air T Sin −T uf Server Ideal 0
sup
(2009) ratio air which goes temperature, floor void P¼ level Good b0.2
T Sout −T uf
sup
into the IT temperature
equipment
Tozer et al. BAL Balance ratio The difference Server inlet and outlet air T S −T S Room Ideal 1
(2009) between cold temperature, return air BAL ¼ Cout Cin level
T ret −T sup
air produced temperature to CRAC,
by the cooling discharge air temperature
plant and the from CRAC
server request
Qian et al. AMI Air Mixing The effects of Thermal resistance at Rsys −Rnm Room Optimal 0
AMI ¼
(2013) Index air mixing on actual operating Rsys level
the data center condition and no air
cooling system mixing condition
Qian et al. ADI Air Thermal Thermal resistance at no Ropt Room Optimal 1
ADI ¼
(2013) Distribution resistance ratio air mixing condition and Rnm level
Index of the optimal no air mixing +optimal
condition to air distribution condition
the no mixing
condition
Qian et al. IHTI Integrated Assess the data Thermal resistance at Ropt Room Optimal 1
IHTI ¼ ð1−AMIÞ∙ADI ¼
(2013) Heat Transfer center heat actual operating Rsys level
Index transfer condition and no air
efficiency mixing +optimal air
relative to the distribution condition
optimal
condition

severity of air bypassing and recycling. An RTI N 100% signifies recircula-
tion, and an RTI b 100% signifies bypassing. VanGilder and Shrivastava
(2006, 2007) proposed the capture index (CI) and the recirculation
index (RI) to capture the cooling performance based solely on the air-
flow associated with the supply of cooling air to, or the removal of hot
air from, a rack. High CI and low RI values (CI + RI = 1) typically
imply good cooling performance.
Some other metrics, such as negative pressure (NP), bypass (BP), re-
circulation (R) and balance (BAL) were introduced by Tozer et al. (2009)
to quantify the balance of the CRAC and server air and airflow distribu-
tion in the data center. In addition, from the aspect of entransy dissipa-
tion analysis, the air mixing index (AMI) can be used to evaluate the
effects of air mixing on the data center cooling system. A higher AMI in-
dicates that serious air mixing strongly influences the cooling system.
The air distribution index (ADI) is used to evaluate the effects of ineffec-
tive cooling air distribution on the data center cooling system. More-
over, the integrated heat transfer index (IHTI), which is equal to the
product of 1-AMI and ADI, was proposed to indicate the potential for
improvement of the data center airflow organization. A higher value
of the IHTI is a result of a smaller AMI and a larger ADI, which indicates
a better cooling air distribution in the data center (Qian et al., 2013). The
explanation of these metrics is listed in Table 3.
In the evaluation of airflow distribution, it is the ideal state that the
all the chilled air can be used by the IT equipment. In other words, if
the airflow rate of the CRACs is equal to that of the total airflow rate of
servers or racks, the airflow can be regarded as good. However, this is
impossible in practice owning to the air leakage, bypassing or recircula-
tion. Then, the efficient cooling becomes the main evaluation of the air-
flow distribution. Also, the magnitude of the air mixing can indirectly
illustrate how much the chilled air is being used effectively. Hence, the
above performance metrics can be roughly divided into three catego-
ries: (1) quantify the balance of the CRAC and server air; (2) assess
the magnitude of recirculation and mixing of hot and cold streams;
(3) address the efficient cooling. As shown in Table 4, most of the per-
formance metrics are not easily acquired through measuring experi-
mentally. As for the metrics to quantify the balance of the CRAC and
server air, with more and more airflow strategies applied in data center,
it is impossible to easily obtain these metrics only through measuring
experimentally. As for the metrics to assess the magnitude of recircula-
tion and mixing of hot and cold air, the HMI and HHMI are seen to be the
best choice, because they not only evaluate the internal thermal envi-
ronment of the rack, but also indicate the hot spot of the rack. However,
it takes a lot of effort and is difficult to obtain the highest and lowest
inlet air temperature of a rack. Hence, considering the convenience in
practice, SHI or RHI combining with β index can evaluate the whole
thermal environment and localized nature of data center hot spot. As
for the metrics to address the efficient cooling, RCI and RTI are recom-
mended. This is because they not only ensure the inlet air temperature
within the recommended or allowable temperature range, but also
evaluate whether the IT equipment is overcooled or undercooled.
 C. Jin et al. / Science of the Total Environment 695 (2019) 133801 13

Table 4
Classification of the performance metrics.

Application scenarios Metric Advantages Disadvantages

Quantify the balance of the
CRAC and server air
Assess the magnitude of
recirculation and mixing of
hot and cold streams
Address the efficient cooling
NP Measures the ambient air infiltration into the under-floor They are impossible to be obtained only through measuring
BP Measures the rate of air which doesn't get inside IT experimentally.
equipment
R Indicates the rate of hot air that goes into the IT equipment
BAL Measures the difference between cold air produced by the
cooling plant and the server request
SHI Quantify the degree of mixing that is occurring between the They are largely “global” metrics, and do not characterize the localized
RHI cool supply air and warm return air and can help evaluate the nature of data center hot spot.
design of hot and cold aisles.
β Captures the local hotspots Cannot reflect the airflow organization problems
Index
IOM Characterizes the air mixing of a rack It takes a lot of effort and is difficult to obtain the highest and lowest
HIOM Characterizes the air mixing of all racks considering the heat inlet air temperature of a rack. Also, they do not characterize the
dissipation of each rack localized nature of rack hot spot.
HMI Evaluates the internal thermal environment of the rack It takes a lot of effort and is difficult to obtain the highest inlet air
HHMI Evaluates the internal thermal environment of the racks temperature and lowest inlet air temperature of a rack.
considering the heat dissipation of each rack
CI Optimize the placement of racks and coolers in a hot-aisle They are not easily measured experimentally (although a rough
RI cluster with local coolers estimate may be made through smoke visualization) but is readily
computed based on CFD or similar simulations.
RCI Evaluates whether the temperatures are within the They do not describe the specific location where hot and cold air mixing
recommended or allowable temperature range.
RTI Evaluates whether the IT equipment is overcooled or
undercooled
AMI Evaluates the effects of air mixing on the data center cooling Although they can guide the improvement and optimization of airflow
system organization, they cannot provide which airflow strategy is the best
ADI Evaluates the effects of ineffective cooling air distribution on choice.
data center cooling system
IHTI Indicates the potential for improvement of the data center
airflow organization

4.2. Optimization of airflow distribution achieve a desirable distribution above the floor plane because of the
changing degree of inclination (Joshi and Kumar, 2012). Another way
Considering the abundant energy consumption of the data center, to of achieving uniform flow rate distribution and reducing wasted air is
achieve the goal of energy savings, many technologies and approaches to use tiles with different openings at different locations (see Fig. 6).
are applied in the data center. Attempting to implement such energy- The distribution becomes significantly uniform and reverse flow is elim-
saving measures without fully understanding the thermal environment inated (Patankar, 2008). Also, active tiles or fan-assisted perforated tiles
can have detrimental effects on the reliability and performance of the IT can increase the tile airflow rates and result in full provisioning of the air
equipment (Green et al., 2012). Thus, methods of optimizing the ther- supply. Especially, the use of a contained cold aisle in combination with
mal environment of the data center from the room level to the server a straightened perforation flow can achieve a near-ideal temperature
level are summarized in this part of the paper. distribution in the cold aisle as well as at the server inlets (Arghode
et al., 2016; Song, 2016a, 2016c, 2017a, 2017b). Optimization through
4.2.1. Room level a combination of these causes the average rack inlet temperature
From the existing literature, the room-level optimization of air dis- dropped from 23.2 °C to 18.2 °C, an improvement of 21.6% (Bhopte
tribution can be divided into three categories: cooling type, air supply et al., 2006a). The solar chimney is another ideal and effective way to
mode and air distribution strategy. Raised-floor cooling is a widely improve the thermal environment of the data center. It provides
used cooling strategy, but it may not be effective for some high- power to enhance the air flow, which makes the temperature and air-
density racks. The raised-floor hybrid cooling strategy can solve the flow distribution more homogeneous both at the rack level and the
problem of variation of server power and airflow rate. However, for room level. It is beneficial to install the solar chimney above the cold
nonuniform rack power conditions, an increase of the power of certain
racks might result in poor cooling conditions for other lower power
racks, even though the cooling is increased together (Gao et al., 2016).
Moreover, to optimize the air distribution in the plenum, two air supply
modes, “mixing” strategy and the “fire-fighting” strategy were investi-
gated. In the mixing strategy, the direction of supply air jets is randomly
changed. The fire-fighting strategy detects hot spots under the perfo-
rated tiles and directs the air jets of the nearest CRACs to these hot
spots. Both of these strategies can be effective for removing hot spots
in the plenum with appropriate randomization procedures (Chen
et al., 2006). Plenum depth, ceiling height, and cold aisle location have
a significant effect on the rack inlet air temperatures. To overcome the
uniform air distribution, vertical barriers or flow deflectors can also be
used to divert air through the perforated tiles. The under-floor partitions
can increase the plenum resistance, thereby resulting in a uniform air
distribution. This configuration is illustrated in Fig. 5. The airflow can Fig. 5. The configuration of the under-floor partitions (Joshi and Kumar, 2012).
14 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
Fig. 6. The optimization of the perforated tile arrangement (Patankar, 2008).
aisle, which can decrease the temperature in the upper zone of the cold
aisle by 13 °C (Zhang et al., 2014b).
4.2.2. Rack level
Rack-level airflow optimization is entirely about blocking cold air-
flow bypassing, blocking hot airflow recirculation, or adding flow guid-
ing devices. In a data center, it is almost impossible for the servers to
fulfill the racks, so there are some empty spaces in the racks where
cold air could bypass and hot air could pass through. To prevent these
circumstances, blanking panels to fill in the empty spaces are recom-
mended. These can restrain the mixing of cold and hot air
(Rasmussen, 2011). Similarly, a novel drawer-type rack with a configu-
ration different from that of the conventional racks can provide some
obstacles to recirculation, as shown schematically in Fig. 7. The move-
able and flexible layout of the rack can reduce the bypass cold flow
from the perforated tile. The rack maximum inlet temperature of the
drawer-type rack can be reduced by 13.3 °C compared with the tradi-
tional one (Wang et al., 2015).
In addition, an innovative airflow optimization method of applying
flexible baffles (FBs) set in the air inlet of racks has been proposed to im-
prove the airflow distribution in an aisle-contained data center (Yuan
et al., 2018). Four width and six angle scenarios have been investigated
using experimental and numerical methods. It has been noted that FBs
set in the air inlets of racks improve the thermal environment of racks
with high density. In particular, the temperature drop is greater when
the temperature of the point is higher or when the width of the baffle
is wider. The optimum thermal environment occurs with a 20 cm
width and a 75° angle of the FBs. Moreover, the temperature drop of a
rack hotspot can reach 0.85–1.7 °C. The author also investigated
lower-side terminal baffles (SLTBs) for servers and found that the appli-
cation of 45° and 8 cm SLTBs can decrease the temperature of a rack
hotspot. The temperature drop can reach 1.9–2.5 °C (Yuan et al., 2019).
4.2.3. Server level
At the server level, the server fan determines the thermal environ-
ment. Hence, optimization of the airflow distribution at the server
level is mainly related to the speed of the fan. Usually, dynamic fan
speed control reuses the native fan speed control algorithm but adjusts
Fig. 7. A novel drawer-type rack (Wang et al., 2015).
its setting in response to CPU utilization changes while meeting a CPU
temperature upper bound requirement (Chen et al., 2015). In general,
at any given speed, setting the fan can dissipate only a limited amount
of heat from a CPU. Increasing the speed requires an exponential in-
crease in fan power. This indicates that temperature solutions that act
only within a socket are not sufficient to minimize cooling energy as
some sockets may generate much more heat than others, requiring a
better heat balance between them. Therefore, a multi-tier algorithm
that schedules the workload at the core and socket levels has been pro-
posed minimize cooling energy and the occurrence of thermal emer-
gencies (Ayoub et al., 2011). Improved fan control will result in
significant speed reduction, which translates into huge potential savings
(Mulay, 2010).
Among these three levels of optimization, server-level optimization
is the most unsuitable to a thermal manager for optimizing the thermal
environment, whether in the design and construction stage or the trans-
formation stage, because it belongs within the realm of server thermal
design. However, the designer of a data center could select appropriate
servers to distribute heat sources reasonably. Conversely, rack-level op-
timization is the best choice for thermal management because it is flex-
ible, convenient, and has relatively low cost. Additionally, the blanking
panel, which can enhance the isolation of the cold and hot air, is widely
applied in data centers. Room-level optimization is applicable to the
decision-making stage of new data center construction or to the renova-
tion of an existing data center. Most suitable among all the optimization
measures at the room level, if the operator somewhere finds or adds
power in some racks, he can adjust the air distribution of nearby racks
by changing the characteristics of tiles, such as by changing the open
ratio of perforated tiles or by adding fan-assistant tiles, which has a
rapid effect and short period. Considerations of the type of cooling and
the mode of air supply are suitable for the data center retrofit project.
5. Conclusion and recommendations for future work
The thermal environment plays an important role in the safety of IT
equipment and the energy savings possible in data centers. Ways to re-
duce the impact of various factors on the thermal environment, to pre-
dict and evaluate the thermal environment effectively, and to optimize
the thermal environment of the data center, have become issues that
data center practitioners need to consider carefully at all times in the
entire lifespan of a data center. From this review, the following conclu-
sions and recommendations for further work can be presented:
1) As for the room level, it is recommended that the CRACs locate at the
ends of each rack row could be the optimal choice, and the height of
plenum and ceiling is 0.79–0.91 m and 2.52 m, respectively. Besides,
it is suggested that the open area of the perforated tiles is 25%, and
the airflow outlet angle is 60°. As for the rack level, the minimum
open area of 60% is advised, and the uniform installation of racks is
the best choice except for the high-density rack. However, the inter-
actions among the factors are unknown, and this topic should be-
come an objective of future research to guide engineering practice.
 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
2) With the development of computational technology, numerical sim-
ulation has become an indispensable method in data center design,
operation, and transformation. Comparing existing simulation
tools, 6SigmaDCX is recommended because of its specialty, easy to
use. Besides, in mesh generation, the simplest way is to retain the
Cartesian approach, but to no longer insist on the lines extending
all the way through the solution domain, and the standard k-ε turbu-
lence model with standard wall functions can satisfy the general
model. In addition, a professional should handle such professional
affairs. Otherwise, the simulation results may mislead decision-
making.
3) Performance metrics provide criteria that can be used by operators
in the operation phase of a data center to estimate whether the ther-
mal environment is good. If engineers want to assess the magnitude
of recirculation and mixing of hot and cold air, SHI or RHI combining
with β index can be the best choice. If the engineers want to address
the cooling efficiency, RCI and RTI are recommended. Besides, if the
improvement and optimization of airflow organization are the focus
of the evaluation, IHTI is the optimal choice. In addition, rack-level
optimization is the primary choice, whatever it is a new data center
or a retrofitted data center.
4) The thermal environment plays an important role in the energy con-
sumption of IT equipment and non-IT equipment, e.g., the CRACs.
Higher ambient air temperature may reduce the CRAC load but
also cause the IT equipment to consume more energy. What is the
optimal solution for these? This is undoubtedly a problem that will
require attention and innovative solutions for a long time into the fu-
ture. Furthermore, if the relationship between the thermal environ-
ment and energy consumption is clarified, refined thermal
management is not far away.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influ-
ence the work reported in this paper.
Acknowledgement
The study is supported by the National Natural Science Foundation
of China (51778080).
References
Abdelmaksoud, W.A., Khalifa, H.E., Dang, T.Q., Elhadidi, B., Schmidt, R.R., Iyengar, M., 2010.
Experimental and computational study of perforated floor tile in data centers. 2010
12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena
in Electronic Systems.
Abdelmaksoud, W.A., Dang, T.Q., Khalifa, H.E., Schmidt, R.R., Iyengar, M., 2012. Perforated
tile models for improving data center CFD simulation. 2012 13th IEEE Intersociety
Conference on Thermal and Thermomechanical Phenomena in Electronic Systems
(Itherm), pp. 60–67.
Alissa, H.A., Nemati, K., Sammakia, B., Ghose, K., Seymour, M., Schmidt, R., 2015a. Innova-
tive approaches of experimentally guided CFD modeling for data centers. 2015 31st
Annual Semiconductor Thermal Measurement, Modeling & Management Symposium
(Semi-Therm), pp. 176–184.
Alissa, H.A., Nemati, K., Sammakia, B., Seymour, M., Schneebeli, K., Schmidt, R., 2015b. Ex-
perimental and numerical characterization of a raised floor data center using rapid
operational flow curves model. International Technical Conference and Exhibition
on Packaging and Integration of Electronic and Photonic Microsystems. vol. 1 2015.
Alkharabsheh, S., Sammakia, B., Murray, B., Shrivastava, S., Schmidt, R., 2014a. Experimen-
tal characterization of pressure drop in a server rack. 2014 IEEE Intersociety Confer-
ence on Thermal and Thermomechanical Phenomena in Electronic Systems,
pp. 547–556.
Alkharabsheh, S., Sammakia, B., Shrivastava, S., Schmidt, R., 2014b. Dynamic models for
server rack and CRAH in a room level CFD model of a data center. 2014 IEEE Interso-
ciety Conference on Thermal and Thermomechanical Phenomena in Electronic Sys-
tems (Itherm), pp. 1338–1345.
Alkharabsheh, S.A., Muralidharan, B., Ibrahim, M., Shrivastava, S.K., Sammakia, B.G., 2014c.
Open and contained cold aisle experimentally validated CFD model implementing
Crac and server fan curves for a data center test laboratory. Proceedings of the
ASME International Technical Conference and Exhibition on Packaging and Integra-
tion of Electronic and Photonic Microsystems. vol. 2 2013.
15
Alkharabsheh, S.A., Sammakia, B., Shrivastava, S., Ellsworth, M., David, M., Schmidt, R.,
2014d. A numerical steady state and dynamic study in a data center using calibrated
fan curves for Cracs and servers. Proceedings of the ASME International Technical
Conference and Exhibition on Packaging and Integration of Electronic and Photonic
Microsystems 2, 2013.
Alkharabsheh, S.A., Sammakia, B.G., Shrivastava, S., Schmidt, R., 2014e. A numerical study
for contained cold aisle data center using Crac and server calibrated fan curves. Pro-
ceedings of the ASME International Mechanical Engineering Congress and Exposition
10, 2013.
Alkharabsheh, S.A., Shrivastava, S.K., Sammakia, B.G., 2014f. Effect of cold aisle contain-
ment leakage on flow rates and temperatures in a data center. Proceedings of the
ASME International Technical Conference and Exhibition on Packaging and Integra-
tion of Electronic and Photonic Microsystems. vol. 2 2013.
Arghode, V.K., Joshi, Y., 2013. Modeling strategies for air flow through perforated tiles in a
data center. IEEE Transactions on Components Packaging and Manufacturing Tech-
nology 3, 800–810.
Arghode, V.K., Joshi, Y., 2015. Experimental investigation of air flow through a perforated
tile in a raised floor data center. J. Electron. Packag. 137.
Arghode, V.K., Joshi, Y., 2016. Air Flow Management in Raised Floor Data Centers.
Springer.
Arghode, V.K., Kumar, P., Joshi, Y., Weiss, T., Meyer, G., 2013a. Rack level modeling of air
flow through perforated tile in a data center. J. Electron. Packag. 135.
Arghode, V.K., Sundaralingam, V., Joshi, Y., Phelps, W., 2013b. Thermal characteristics of
open and contained data center cold aisle. Journal of Heat Transfer-Transactions of
the ASME 135.
Arghode, V.K., Sundaralingam, V., Joshi, Y., 2016. Airflow management in a contained cold
aisle using active fan tiles for energy efficient data-center operation. Heat Transfer
Engineering 37, 246–256.
Ayoub, R., Indukuri, K., Rosing, T.S., 2011. Temperature aware dynamic workload schedul-
ing in multisocket CPU servers. IEEE Transactions on Computer-Aided Design of Inte-
grated Circuits and Systems 30, 1359–1372.
Bao, L.L., Wang, J.G., Kang, L.G., 2012. The applied effect analysis of heat exchanger
installed in a typical communication base station in Beijing of China. 2011 2nd Inter-
national Conference on Advances in Energy Engineering (ICAEE). vol. 14,
pp. 620–625.
Bedekar, V., 2006. Effect of CRAC Location on a Fixed Rack Layout of a Data Center, Science
in Mechanical Engineering. The University of Texas at Arlington, p. 63.
Beitelmal, A.H., 2016. Numerical investigation of data center raised-floor plenum. Pro-
ceedings of the ASME International Mechanical Engineering Congress and Exposition,
2015. vol 8a.
Bhopte, S., Agonafer, D., Schmidt, R., Sammakia, B., 2006a. Optimization of data cen-
ter room layout to minimize rack inlet air temperature. J. Electron. Packag. 128,
380–387.
Bhopte, S., Iyengar, M.K., Sammakia, B., Schmidt, R., Agonafer, D., 2006b. Numerical
Modeling of Data Center Clusters: Impact of Model Complexity 2006 ASME
International Mechanical Engineering Congress and Exposition, Chicago,
Illinois, USA.
Bhopte, S., Sammakia, B., Iyengar, M.K., Schmidt, R., 2006c. Guidelines on Managing Under
Floor Blockages for Improved Data Center Performance, 2006 ASME International
Mechanical Engineering Congress and Exposition, Chicago, Illinois, USA.
Bhopte, S., Sammakia, B., Schmidt, R., Iyengar, M.K., Agonafer, D., 2006d. Effect of under
floor blockages on data center performance. 2006 Proceedings 10th Intersociety Con-
ference on Thermal and Thermomechanical Phenomena in Electronics Systems. vols
1 and 2 (426-+).
Breen, T.J., Walsh, E.J., Punch, J., Shah, A.J., Bash, C.E., 2010. From chip to cooling tower data
center modeling: part I. Influence of server inlet temperature and temperature rise
across cabinet. 2010 12th IEEE Intersociety Conference on Thermal and
Thermomechanical Phenomena in Electronic Systems.
C.A.R, D.C.o, 2018. China Data Center Annual Research Report on Cooling Technology De-
velopment 2017. China Architecture & Building Press.
C.A.R, D.C.o, 2019. China Data Center Annual Research Report on Cooling Technology De-
velopment 2017. China Architecture & Building Press.
Carrera, E.V., Pinheiro, E., Bianchini, R., 2003. Conserving disk energy in network servers.
International Conference on Supercomputing, pp. 86–97.
Chen, J., Tan, R., Xing, G., Wang, X., 2015. PTEC: a system for predictive thermal and en-
ergy control in data centers. Real-Time Systems Symposium, pp. 218–227.
Chen, K., Federspiel, C.C., Auslander, D.M., Bash, C.E., Patel, C.D., 2006. Control Strategies
for Plenum Optimization in Raised Floor Data Centers. pp. 1–18.
Cho, J., Yang, J., Park, W., 2014. Evaluation of air distribution system's airflow performance
for cooling energy savings in high-density data centers. Energy and Buildings 68,
270–279.
Choi, J., Kim, Y., Sivasubramaniam, A., Srebric, J., Wang, Q., Lee, J., 2008. A CDF-based tool
for studying temperature in rack-mounted servers. IEEE Trans. Comput. 57,
1129–1142.
Committee, C.D.C., 2017. China Data Center Annual Research Report on Cooling Technol-
ogy Development 2016. China Architecture & Building Press.
Dai, J., Ohadi, M.M., Das, D., Pecht, M.G., 2014. Optimum Cooling of Data Centers. Springer.
Daraghmeh, H.M., Wang, C.C., 2017. A review of current status of free cooling in
datacenters. Appl. Therm. Eng. 114, 1224–1239.
Ding, T., He, Z.G., Hao, T., Li, Z., 2016. Application of separated heat pipe system in data
center cooling. Appl. Therm. Eng. 109, 207–216.
Dunlap, K., Rasmussen, N., 2019. Choosing Between Room, Row, and Rack-based Cooling
for Data Centers (White paper).
Ebrahimi, K., Jones, G.F., Fleischer, A.S., 2014. A review of data center cooling technology,
operating conditions and the corresponding low-grade waste heat recovery opportu-
nities. Renew. Sustain. Energy Rev. 31, 622–638.
16 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
Fakhim, B., Srinarayana, N., Behnia, M., Armfield, S.W., 2013. Thermal performance of data
centers-rack level analysis. IEEE Transactions on Components Packaging and
Manufacturing Technology 3, 792–799.
Fulpagare, Y., Bhargav, A., 2015. Thermal prediction model of data center. Proceedings of
the 23rd National Heat and Mass Transfer Conference and 1st International ISHMT-
ASTFE Heat and Mass Transfer Conference, Thiruvananthapuram, India.
Fulpagare, Y., Mahamuni, G., Bhargav, A., 2015. Effect of plenum chamber obstructions on
data center performance. Appl. Therm. Eng. 80, 187–195.
Fulpagare, Y., Joshi, Y., Bhargav, A., 2018. Rack level transient CFD modeling of data center.
International Journal of Numerical Methods for Heat & Fluid Flow 28, 381–394.
Gao, T.Y., Geer, J., Sammakia, B.G., Tipton, R., Seymour, M., 2016. Perimeter cooling unit
and localized row-based cooling unit transient air flow effects modeling and charac-
terization in data centers. Proceedings of the ASME International Mechanical Engi-
neering Congress and Exposition, 2015. vol. 8b.
Garraghan, P., Al-Anii, Y., Summers, J., Thompson, H., Kapur, N., Djemame, K., 2016. A uni-
fied model for holistic power usage in cloud datacenter servers. 2016 IEEE/ACM 9th
International Conference on Utility and Cloud Computing (UCC) 11–19.
Geng, H., 2015. Data Center Handbook. John Wiley & Sons.
Ghosh, R., Sundaralingam, V., Joshi, Y., 2012. Effect of rack server population on tempera-
tures in data centers. 2012 13th IEEE Intersociety Conference on Thermal and
Thermomechanical Phenomena in Electronic Systems (Itherm), pp. 30–37.
Gondipalli, S., Bhopte, S., Sammakia, B., Iyengar, M.K., Schmidt, R., 2008. Effect of isolating
cold aisles on rack inlet temperature. 2008 11th IEEE Intersociety Conference on
Thermal and Thermomechanical Phenomena in Electronic Systems. vols. 1-3
(1247-+).
Green, M., Karajgikar, S., Vozza, P., Gmitter, N., Dyer, D., 2012. Achieving energy efficient
data centers using cooling path management coupled with ASHRAE standards.
2012 28th Annual IEEE Semiconductor Thermal Measurement and Management
Symposium (Semi-Therm) (288−+).
Gurumurthi, S., Sivasubramaniam, A., Kandemir, M., Franke, H., 2003. DRPM: dynamic
speed control for power management in server class disks. 30th Annual International
Symposium on Computer Architecture, Proceedings, pp. 169–179.
Ham, S.W., Kim, M.H., Choi, B.N., Jeong, J.W., 2015. Simplified server model to simulate
data center cooling energy consumption. Energy and Buildings 86, 328–339.
Hamann, H., Iyengar, M., O'Boyle, M., 2008. The impact of air flow leakage on server inlet
air temperature in a raised floor data center. 2008 11th IEEE Intersociety Conference
on Thermal and Thermomechanical Phenomena in Electronic Systems. vols 1-3
(1153-+).
Han, L.J., Shi, W.X., Wang, B.L., Zhang, P.L., Li, X.T., 2013. Development of an integrated air
conditioner with thermosyphon and the application in mobile phone base station. In-
ternational Journal of Refrigeration-Revue Internationale Du Froid 36, 58–69.
Hassan, S.F., Ali, M., Perwez, U., Sajid, A., 2015. Free cooling investigation of RCMS
data center. Clean, Efficient and Affordable Energy for a Sustainable Future 75,
1249–1254.
Herrlin, M.K., 2005. Rack cooling effectiveness in data centers and telecom central offices:
the rack cooling index (RCI). ASHRAE Trans. 111 (Pt 2 111), 725–731 2005.
Herrlin, M.K., 2008. Airflow and cooling performance of data centers: two performance
metrics. ASHRAE Trans. 114, 182–187 Pt 2 114.
http://tileflow.com/tileflow/data-center-cfd-modeling-software-overview.html.
http://www.thefreelibrary.com/Effective+computer+room+cooling.+%28Infrastruc-
ture%29.-a0106646787.
https://www.futurefacilities.com/.
https://www.mentor.com/products/mechanical/flotherm/.
https://www.mentor.com/products/mechanical/flovent/.
Ibrahim, M., Bhopte, S., Sammakia, B., Murray, B., Iyengar, M., Schmidt, R., 2012a. Effect of
transient boundary conditions and detailed thermal modeling of data center rooms.
IEEE Transactions on Components Packaging and Manufacturing Technology 2,
300–310.
Ibrahim, M., Shrivastava, S., Sammakia, B., Ghose, K., 2012b. Thermal mass characteriza-
tion of a server at different fan speeds. 2012 13th IEEE Intersociety Conference on
Thermal and Thermomechanical Phenomena in Electronic Systems (Itherm),
pp. 457–465.
Iyengar, M., Schmidt, R., Sharma, A., McVicker, G., Shrivastava, S., Sri-Jayantha, S.,
Amemiya, Y., Dang, H., Chainer, T., Sammakia, B., 2005. Thermal characterization of
non-raised floor air cooled data centers using numerical modeling. Advances in Elec-
tronic Packaging 2005, Pts A-C, pp. 535–543.
Iyengar, M., Schmidt, R.R., Hamann, H., VanGilder, J., 2007. Comparison between nu-
merical and experimental temperature distributions in a small data center test
cell. Ipack 2007: Proceedings of the ASME Interpack Conference 2007. vol 1,
pp. 819–826.
Iyengar, M., Schmidt, R., Kamath, V., Kochuparambil, B., 2012. Experimental characteriza-
tion of server rack energy use at elevated ambient temperatures. Proceedings of the
ASME Pacific Rim Technical Conference and Exhibition on Packaging and Integration
of Electronic and Photonic Systems, Mems and Nems 2011. vol. 2 (617-+).
Joshi, Y., Kumar, P., 2012. Energy Efficient Thermal Management of Data Centers. Springer.
Karki, K., 2007. Prediction of distributed air leakage in raised floor data centers (DA-07-
023). Ashrae Winter Meeting, pp. 217–226.
Karki, K.C., Patankar, S.V., Radmehr, A., 2003. Techniques for controlling airflow distribu-
tion in raised-floor data centers. Advances in Electronic Packaging 2003. vol. 2,
pp. 621–628.
Khankari, K., 2014. Analysis of air leakage from hot aisle containment systems and cooling
efficiency of data centers. ASHRAE Trans. 120 (Pt 1), 120 2014.
King, D., Ross, M., Seymour, M., Gregory, T., 2014. Comparative analysis of data center de-
sign showing the benefits of server level simulation models. 2014 30th Annual Semi-
conductor Thermal Measurement and Management Symposium (Semi-Therm),
pp. 193–196.
Kumar, P., Sundaralingam, V., Joshi, Y., 2011. Effect of server load variation on rack air
flow distribution in a raised floor data center. 2011 27th Annual IEEE Semicon-
ductor Thermal Measurement and Management Symposium (Semi-Therm),
pp. 90–96.
Lee, K.P., Chen, H.L., 2013. Analysis of energy saving potential of air-side free cooling for
data centers in worldwide climate zones. Energy and Buildings 64, 103–112.
Li, L.C., Liu, Y.L., Wang, X.L., Nnanna, A.G.A., 2010. Investigation of indoor environment for
a data center. Imece 2009: Proceedings of the ASME International Mechanical Engi-
neering Congress and Exposition. vol. 9, pp. 633–642 Pts a-C.
Ling, Y.Z., Zhang, X.S., Li, S.H., Han, G.N., Sun, X.G., 2015. The numerical simulation and ex-
perimental study on rack cooling effect in date center with UFAD system. Interna-
tional Journal of Low-Carbon Technologies 10, 446–451.
Ling, Y.Z., Zhang, X.S., Zhang, K., Jin, X., 2017. On the characteristics of airflow through the
perforated tiles for raised-floor data centers. Journal of Building Engineering 10,
60–68.
Liu, S., Zhang, H., Huang, L., Chen, W., Li, M., Huang, H., Chen, H., Xie, C., Wang, P., 2015.
Experimental study on application of heat pipe backboard air-conditioner in data
centers-analysis of air flow field in computer room. Refrigeration and Air-
conditioning 15, 51–56.
Lu, H.J., Zhang, Z.B., Yang, L., 2018. A review on airflow distribution and management in
data center. Energy and Buildings 179, 264–277.
Marcinichen, J.B., Thome, J.R., 2012. Two-phase flow control of on-Chip two-phase cooling
systems of servers. 2012 28th Annual IEEE Semiconductor Thermal Measurement
and Management Symposium (Semi-Therm), pp. 29–35.
Marcinichen, J.B., Olivier, J.A., Thome, J.R., 2012. On-chip two-phase cooling of
datacenters: cooling system and energy recovery evaluation. Appl. Therm. Eng. 41,
36–51.
Moss, D., Bean, J.H., 2010. Energy Impact of Increased Server Inlet Temperature (White
paper).
Mulay, V.P., 2010. Analysis of Data Center Cooling Strategies and the Impact of the Dy-
namic Thermal Management on the Data Center Efficiency. Faculty of Graduate
School. The University of Texas at Arlington.
Nada, S.A., Elfeky, K.E., 2016. Experimental investigations of thermal managements solu-
tions in data centers buildings for different arrangements of cold aisles containments.
Journal of Building Engineering 5, 41–49.
Nada, S.A., Elfeky, K.E., 2017. Effects of servers' rack location and power loading configu-
rations on the thermal management of data center racks' array. Journal of Thermal
Science and Engineering Applications 9.
Nada, S.A., Said, M.A., 2017. Effect of CRAC units layout on thermal management of data
center. Appl. Therm. Eng. 118, 339–344.
Nada, S.A., Elfeky, K.E., Attia, A.M.A., 2016a. Experimental investigations of air condition-
ing solutions in high power density data centers using a scaled physical model. Inter-
national Journal of Refrigeration-Revue Internationale Du Froid 63, 87–99.
Nada, S.A., Said, M.A., Rady, M.A., 2016b. CFD investigations of data centers' thermal per-
formance for different configurations of CRACs units and aisles separation. Alexandria
Engineering Journal 55, 959–971.
Nagarathinam, S., Fakhim, B., Behnia, M., Armfield, S., 2013. A comparison of parametric
and multivariable optimization techniques in a raised-floor data center. J. Electron.
Packag. 135.
Nemati, K., Alissa, H., Sammakia, B., 2016. Performance of temperature controlled perim-
eter and row-based cooling systems in open and containment environment. Proceed-
ings of the ASME International Mechanical Engineering Congress and Exposition,
2015. vol. 8b.
Ni, J., Jin, B.W., Zhang, B., Wang, X.W., 2017. Simulation of thermal distribution and airflow
for efficient energy consumption in a small data centers. Sustainability 9.
Ni, J.C., Bai, X.L., 2017. A review of air conditioning energy performance in data centers.
Renew. Sustain. Energy Rev. 67, 625–640.
Pardey, Z.M., VanGilder, J.W., Healey, C.M., Plamondon, D.W., 2015. Creating a calibrated
CFD model of a midsize data center. International Technical Conference and Exhibi-
tion on Packaging and Integration of Electronic and Photonic Microsystems, 2015.
vol. 1.
Patankar, S., 2008. CFD simulation of airflow and cooling in a data center. Cht-08 Ichmt
International Symposium on Advances in Computational Heat Transfer, pp. 1–8.
Patel, C.D., Bash, C.E., Belady, C., 2001. Computational fluid dynamics modeling of high
compute density data centers to assure system inlet air specifications. The Pacific
Rim/ASME International Electronic Packaging Technical Conference and Exhibition
Kauai, Hawaii, USA.
Phan, L., Hu, B.C., Lin, C.X., 2019. An evaluation of turbulence and tile models at server rack
level for data centers. Build. Environ. 155, 421–435.
Priyadumkol, J., Kittichaikarn, C., 2013. A study of air flow through perforated tile for air
conditioning system in data center. Applied Mechanics & Materials 126–131.
Qian, X.D., Li, Z., Li, Z.X., 2013. A thermal environmental analysis method for data centers.
Int. J. Heat Mass Transf. 62, 579–585.
Radmehr, A., Karki, K.C., Patankar, S.V., Schmidt, R.R., 2005. Distributed leakage flow
in raised-floor data centers. Advances in Electronic Packaging 2005 (Pts A-C),
401–408.
Radmehr, A., Noll, B., Fitzpatrick, J., Karki, K., 2013. CFD modeling of an existing raised-
floor data center. 2013 Twenty Ninth Annual IEEE Semiconductor Thermal Measure-
ment and Management Symposium (Semi-Therm), pp. 39–44.
Rambo, J., Joshi, Y., 2005. Thermal performance metrics for arranging forced air cooled
servers in a data processing cabinet. J. Electron. Packag. 127, 452–459.
Rambo, J., Nelson, G., Joshi, Y., 2007. Airflow distribution through perforated tiles in close
proximity to computer room air-conditioning units. ASHRAE Trans. 113 (Pt 2 113),
124–135 2007.
Rasmussen, N., 2011. Improving Rack Cooling Performance Using Airflow Management™.
Blanking Panels APC.
 C. Jin et al. / Science of the Total Environment 695 (2019) 133801
Rodgers, P., Eveloy, V., 2003. Prediction of microelectronics thermal behavior in electronic
equipment: status, challenges and future requirements. Journal of Microelectronics &
Electronic Packaging 1.
Romadhon, R., Ali, M., Mahdzir, A.M., Abakr, Y.A., 2009. Optimization of cooling systems in
data centre by computational fluid dynamics model and simulation. 2009 Conference
on Innovative Technologies in Intelligent Systems and Industrial Applications,
pp. 322–327.
Sakaino, H., 2014. Local and global dimensional CFD simulations and analyses to optimize
server-fin design for improved energy efficiency in data centers. 2014 IEEE Intersoci-
ety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems
(Itherm), pp. 557–567.
Schmidt, R., Cruz, E., 2003. Raised floor computer data center: effect on rack inlet temper-
atures when rack flowrates are reduced. Advances in Electronic Packaging 2,
495–508 2003.
Schmidt, R., Cruz, E., 2004. Cluster of high-powered racks within a raised-floor computer
data center: effect of perforated tile flow distribution on rack inlet air temperatures.
J. Electron. Packag. 126, 510–518.
Schmidt, R., Iyengar, M., 2005. Effect of data center layout on rack inlet air temperatures.
Advances in Electronic Packaging 2005, Pts A-C, pp. 517–525.
Schmidt, R., Karki, K., Patankar, S., 2004. Raised-floor data center: perforated tile flow
rates for various tile layouts. Itherm 1, 571–578 2004.
Schmidt, R., Vallury, A., Iyengar, M., 2012. Energy savings through hot and cold aisle con-
tainment configurations for air cooled servers in data centers. Proceedings of the
ASME Pacific Rim Technical Conference and Exhibition on Packaging and Integration
of Electronic and Photonic Systems, Mems and Nems 2011. vol. 2 (611-+).
Schmidt, R.R., Karki, K.C., Kelkar, K.M., Radmehr, A., Patankar, S.V., 2001. Measurements
and predictions of the flow distribution through perforated tiles in raised-floor data
centers. The Pacific Rim/ASME International Electronic Packaging Technical Confer-
ence and Exhibition, Hawaii, USA, 1–10.
Schmidt, R.R., Cruz, E.E., Iyengar, M.K., 2005. Challenges of data center thermal manage-
ment. IBM J. Res. Dev. 49, 709–723.
Seymour, M., 2015. Modeling perforated tiles in data centers - what is required? 2015
31st Annual Semiconductor Thermal Measurement, Modeling & Management Sym-
posium (Semi-Therm), pp. 8–12
Sharma, R.K., Bash, C.E., Patel, C.D., 2002. Dimensionless parameters for evaluation of ther-
mal design and performance of large-scale data centers. 8th AIAA/ASME Joint
Thermophysics and Heat Transfer Conference. AIAA.
Shrivastava, S.K., 2008. Cooling Analysis of Data Centers CFD Modeling and Real-Time Cal-
culators. Philosophy in Mechanical Engineering Binghamton University.
Shrivastava, S.K., Calder, A.R., Ibrahim, M., 2012. Quantitative comparison of air contain-
ment systems. 2012 13th IEEE Intersociety Conference on Thermal and
Thermomechanical Phenomena in Electronic Systems (Itherm), pp. 68–77.
Shu, Q., 2014. Energy Consumption and Air Distribution Simulation of a Substation Data
Room. Institute of Refrigeration and Cryogenics. Zhejiang University.
Singh, U., Singh, A.K., S, P., Sivasubramaniam, A., 2011. CFD-based Operational Thermal Ef-
ficiency Improvement of a Production Data Center.
Siriwardana, J., Halgamuge, S.K., Scherer, T., Schott, W., 2012. Minimizing the thermal im-
pact of computing equipment upgrades in data centers. Energy and Buildings 50,
81–92.
Song, Z.H., 2016a. Numerical cooling performance evaluation of fan-assisted perforations
in a raised-floor data center. Int. J. Heat Mass Transf. 95, 833–842.
Song, Z.H., 2016b. Numerical study for data center cooling using active tiles. Proceedings
of the ASME International Mechanical Engineering Congress and Exposition, 2015.
vol. 8b.
Song, Z.H., 2016c. Thermal performance of a contained data center with fan-assisted per-
forations. Appl. Therm. Eng. 102, 1175–1184.
Song, Z.H., 2017a. Numerical investigation for performance indices and categorical de-
signs of a fan-assisted data center cooling system. Appl. Therm. Eng. 118, 714–723.
Song, Z.H., 2017b. Studying the fan-assisted cooling using the Taguchi approach in open
and closed data centers. Int. J. Heat Mass Transf. 111, 593–601.
Song, Z.H., Murray, B.T., Sammakia, B., 2014a. Improved zonal model for data center anal-
ysis. Proceedings of the ASME International Technical Conference and Exhibition on
Packaging and Integration of Electronic and Photonic Microsystems. vol. 2 2013.
Song, Z.H., Murray, B.T., Sammakia, B., 2014b. Prediction of hot aisle partition airflow
boundary conditions. Proceedings of the ASME International Technical Conference
and Exhibition on Packaging and Integration of Electronic and Photonic
Microsystems, 2013. vol. 2.
Srinarayana, N., Fakhim, B., Behnia, M., Armfield, S.W., 2014. Thermal performance of an
air-cooled data center with raised-floor and non-raised-floor configurations. Heat
Transfer Engineering 35, 384–397.
Stavreva, S., Dimitrieska, C., Hristovska, E., Andreevski, I., 2016. Study of a data center and
creation of its numerical model. International Journal of Scientific & Engineering Re-
search 7, 1156–1160.
Takahashi, M., Uekusa, T., Kishita, M., Kaneko, H., 2008. Aisle-capping method for airflow
design in data centers. Intelec 08 - 30th International Telecommunications Energy.
vols 1 and 2, pp. 202–208.
17
Tan, S.P., Toh, K.C., Wong, Y.W., 2007. Server-rack air flow and heat transfer interactions in
data centers. Ipack 2007: Proceedings of the ASME Interpack Conference 2007. vol. 1,
pp. 845–849.
Tian, H., Li, Z., Liu, X., Qian, X., 2012. Study on thermal environment evaluation in data
centers. Journal of Refrigeration 33, 5–9.
Tozer, R., Kurkjian, C., Salim, M., 2009. Air management metrics in data centers. ASHRAE
Trans. 115 (Pt 1 115, 63−+).
Vallury, A., Steinke, M.E., Kamath, V., Parnell, L., 2015. A computational study to compare
the data center cooling energy spent using traditional air cooled and hybrid water
cooled servers. International Technical Conference and Exhibition on Packaging and
Integration of Electronic and Photonic Microsystems 1, 2015.
VanGilder, J.W., Schmidt, R.R., 2005. Airflow uniformity through perforated tiles in a
raised-floor data center. Advances in Electronic Packaging 2005, Pts A-C, pp. 493–501.
VanGilder, J.W., Shrivastava, S.K., 2006. Real-time prediction of rack-cooling performance.
ASHRAE Trans. 112 (Pt 2 112) 2006. (151-+).
VanGilder, J.W., Shrivastava, S.K., 2007. Capture index: an airflow-based rack cooling per-
formance metric. ASHRAE Trans. 113, 126–136.
VanGilder, J.W., Sheffer, Z.R., Zhang, X., O'Kane, C.T., 2012. The effect of under-floor ob-
structions on data center perforated tile airflow. Proceedings of the ASME Pacific
Rim Technical Conference and Exhibition on Packaging and Integration of Electronic
and Photonic Systems, Mems and Nems 2011. vol 2 (505-+).
Wang, C.H., Tsui, Y.Y., Wang, C.C., 2017. On cold-aisle containment of a container
datacenter. Appl. Therm. Eng. 112, 133–142.
Wang, I.N., Tsui, Y.Y., Wang, C.C., 2015. Improvements of airflow distribution in a con-
tainer data center. Clean, Efficient and Affordable Energy for a Sustainable Future.
vol. 75, pp. 1819–1824.
Wang, J.Q., Zhang, Q., Yu, Y.B., Chen, X.M., Yoon, S.M., 2018. Application of model-based
control strategy to hybrid free cooling system with latent heat thermal energy stor-
age for TBSs. Energy and Buildings 167, 89–105.
Wang, X.D., Wang, X.R., Xing, G.L., Chen, J.Z., Lin, C.X., Chen, Y.X., 2011. Towards optimal
sensor placement for hot server detection in data centers. 31st International Confer-
ence on Distributed Computing Systems (ICDCS 2011), pp. 899–908.
Webb, M., 2008. Smart 2020: Enabling the Low Carbon Economy in the Information Age.
www.199it.com/archives/670459.html.
Yang, M.C., 2000. A comparison of using icepak and flotherm in electronic cooling. Itherm
2000: Seventh Intersociety Conference on Thermal and Thermomechanical Phenom-
ena in Electronic Systems. vol I, pp. 240–246 Proceedings.
Yuan, X., Wang, Y., Liu, J., Xu, X., Yuan, X., 2018. Experimental and numerical study of air-
flow distribution optimisation in high-density data centre with flexible baffles. Build.
Environ. 140, 128–139.
Yuan, X.L., Zhou, X.T., Liu, J.X., Wang, Y., kosonen, R.O., Xu, X.J., 2019. Experimental and
numerical investigation of an airflow management system in data center with
lower-side terminal baffles for servers. Build. Environ. 155, 308–319.
Yue, C., Zhang, Q., Zhai, Z.Q., Ling, L., 2018. CFD simulation on the heat transfer and flow
characteristics of a microchannel separate heat pipe under different filling ratios.
Appl. Therm. Eng. 139, 25–34.
Zahn, B.A., 1998. Evaluating thermal characterization accuracy using CFD codes - a pack-
age level benchmark study of IcePak (TM) and flotherm (R). Itherm '98: Sixth Inter-
society Conference on Thermal and Thermomechanical Phenomena in Electronic
Systems, pp. 322–329.
Zhai, J.Z., Hermansen, K.A., Al-Saadi, S., 2012. The development of simplified rack bound-
ary conditions for numerical data center models. ASHRAE Trans. 118, 436–449 Pt 2
118.
Zhang, H.N., Shao, S.Q., Xu, H.B., Zou, H.M., Tian, C.Q., 2014a. Free cooling of data centers: a
review. Renew. Sustain. Energy Rev. 35, 171–182.
Zhang, H.N., Shao, S.Q., Xu, H.B., Zou, H.M., Tian, C.Q., 2015. Integrated system of mechan-
ical refrigeration and thermosyphon for free cooling of data centers. Appl. Therm.
Eng. 75, 185–192.
Zhang, K., Zhang, X.S., Li, S.H., Wang, G., 2014b. Numerical study on the thermal environ-
ment of UFAD system with solar chimney for the data center. Proceedings of the 2nd
International Conference on Solar Heating and Cooling for Buildings and Industry
(SHC 2013). vol. 48, pp. 1047–1054.
Zhang, K., Zhang, X., Li, S., 2016. Optimization on airflow distribution in data room air-
conditioning system with underfloor air distribution. Journal of Southeast University
(Natural Science Edition) 46, 62–69.
Zhang, X.H., Iyengar, M., VanGilder, J.W., Schmidt, R.R., 2008. Effect of rack modeling detail
on the numerical results of a data center test cell. 2008 11th IEEE Intersociety Confer-
ence on Thermal and Thermomechanical Phenomena in Electronic Systems. vols 1–3
(1183-+).
Zhu, Y., Liu, J., Wei, L., 2015. Influence of Servers' arrangement on cooling effect in data
center. Building Energy Efficiency 81–84.