Applied Thermal Engineering 164 (2020) 114444

Contents lists available at ScienceDirect

Applied Thermal Engineering

journal homepage: www.elsevier.com/locate/apthermeng

Design and validation of an airflow management system in data center with T

tilted server placement
Xiaolei Yuana,b, Xinjie Xua, Yu Wanga, Jinxiang Liua, , Risto Kosonena,b, Hao Caia
⁎

a
Department of HVAC, College of Urban Construction, Nanjing Tech University, P.O. Box 76, 210009 Nanjing, China
b
Department of Mechanical Engineering, School of Engineering, Aalto University, P.O. Box 14400, FI-00076 Aalto, Finland

HIGHLIGHTS

• An innovative airflow management method of tilting servers is put forward.

• Four different tilted angles (15°, 30°, 45° and 60°) are applied and analyzed.
• The optimum thermal distribution appears with 30° tilted servers in horizontal direction.
• Rack hotspots are weakened significantly minimizing the possibility of local hotspot.
• The application of 30° tilted servers can take away extra 5.7–10% server heat.

ARTICLE INFO ABSTRACT

Keywords: An innovative airflow optimisation method applying tilted servers was introduced to improve the airflow dis-
Data centers tribution and cooling efficiency in data centers. The numerical data center model was established and simulated
Airflow management by Airpak 3.0 packages and validated with on-site measurements in Nanjing, China. In the analysis, different
Tilted servers server inclination angles and different heat load levels were simulated. The simulation model is validated well
Temperature distribution
with the on-site experimental results. The results show that the performance of airflow distribution and cooling
Rack hotspot
Local hotspot
efficiency can be improved with the application of 15° and 30° tilted servers, while the optimum thermal en-
vironment is achieved when the angle of servers is set to 30°. When the heat load of rack is 32 kW, the maximum
temperature reduces by 3.3 °C when the 30° tilted servers are used instead of 0° tilted servers. That decreases
significantly the temperature of rack hotspots and thus prevents local hotspots. Moreover, the heat transfer from
the rack is at least 5.7% more efficient with the rack applying 30° tilted servers compared with the 0° tilted
servers in all cases with different heat load. The heat transfer even improved 10% when the heat load is 32 kW in
each rack.

1. Introduction
Data centers (DCs) are centralized repositories housing computer
servers and associated data storage and telecommunications system for
data storage as well as data processing and transmitting. DCs have the
characteristics of large scale and working 24/24, 365 days/year causing
potentially very high energy consumption [1]. During the past several
decades, in order to satisfy the increasing demand of cloud computing,
data storage and Internet use, the Information and Communications
Technologies (ICTs) experienced a rapid development bringing ex-
plosive energy consumption in DCs [2–4]. In fact, approximately 1.5%
of power generation of the whole world is consumed by DCs in 2010
and the annual growth rate of energy consumption in DCs is around
12% [5,6]. In China, the power consumed by DCs in 2014 reached more
than 83 billion kWh, while the estimated DCs electricity consumption
will increase to approximately 140 billion kWh by 2020 [7,8]. Thus,
under the existing technology conditions, the increase of energy con-
sumption of DCs will be no doubt explosive in the near future.
DCs are building synthesis equipped with many different instru-
ments mainly divided into four parts: IT equipment, cooling system,
power supply equipment and safety & control systems. Fig. 1 illustrates
the composition of typical DCs [9,10]. Among them, IT equipment and
cooling system are two main parts occupying more than 80% of the
energy consumption within DCs [8]. Huge amount of data processing
and circulation results in tremendous electricity consumption, and a
large number of heats converted from the electricity, which needs to be

⁎
Corresponding author.
E-mail address: jxliu@njut.edu.cn (J. Liu).

https://doi.org/10.1016/j.applthermaleng.2019.114444
Received 22 May 2019; Received in revised form 20 August 2019; Accepted 24 September 2019
Available online 25 September 2019
1359-4311/ © 2019 Elsevier Ltd. All rights reserved.
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Nomenclature servers, °C
t1 temperature difference between mean exhaust and supply
P static pressure air temperature with horizontal servers, K
T stress tensor tex mean exhaust air temperature with horizontal servers, °C
keff thermal conductivity ts mean supply air temperature with horizontal servers, °C
g gravitational acceleration vector r ratio of the heat removal ration between with 0° and 30°
S volumetric heat generation tilted servers, %
veff effective fluid viscosity. Q0 heat removal with 30° tilted servers in each case, kJ/h
U0 inlet airflow velocity, m/s Q1 heat difference taken away in the rack with horizontal
Qt tile flow rate, m3/s servers (0°), kJ/h
T0 inlet air temperature, °C
Qs server flow rate, m3/s Abbreviation
Ps server heat generation, W
σt perforated tiles porosity, % DC data center
σr rack porosity, % ICT Information and Communications Technology
Q generated heat, kJ/h CRACs computer room air conditioning units
cp specific heat capacity of air, kJ/(kg K) FB flexible baffle
m mass of air, kg CFD Computational Fluid Dynamics
t temperature difference, K CAC closed aisle containment
air density, kg/m3 SAT supply air temperature
v volume of air, m3 SST Shear Stress Transport
t0 exhaust mean temperature reduction with 0° and 30° tilted RSM Reynolds Stress Model
servers, K DES Detached Eddy Simulations
t 30° exhaust mean temperature of rack A8 with 30° tilted ser- RANS Reynolds-averaged Naiver-Stokes
vers, °C ER experimental result
t 0° exhaust mean temperature of rack A8 with horizontal SR simulation result

dissipated immediately. Otherwise, the temperature of computer ser-
vers may rise dramatically threatening the operation safety of computer
servers and malfunction [11,12]. Fig. 2 shows the breakdown of the
energy consumption within typical DCs, and the IT equipment and the
cooling requirements account for 44% and 40% of the total energy use,
respectively [13,14]. Thus, energy conservation in DCs can be achieved
by addressing IT load inefficiencies and improving cooling efficiency
[13]. It is reported that enhanced semiconductor technologies and
server virtualisation can efficiently decrease the energy use of IT
equipment [15], while the cooling efficiency in DCs can be improved in
many ways, such as airflow management, liquid cooling method, free
cooling technologies, etc. [16]. At the moment, airflow management is
considered as the most adopted method of improving energy efficiency
in DCs. According to Cho et al. [17], airflow management is an operable
and convenient way to reduce energy consumption and enhance
cooling efficiency and the thermal environment in DCs by changing
computer room air conditioning units’ (CRACs) air supply conditions
and configuration of DCs.
Investigation on airflow management methods are an upsurge in the
field of energy conservation in DCs. According to Tao et al. [18], air-
flow management is defined as the way to maintain operating para-
meters (IT equipment) within the recommended ranges with minimum
energy consumption. In recent years, researchers still have efforts on
the airflow distribution optimisation methods to improve the thermal
environment and achieve considerable energy saving within DCs. Yuan
et al. [16] proposed an innovative airflow organisation optimisation
method applying flexible baffles (FBs) set in the front door of racks to
improve cooling efficiency and achieve energy conservation in DCs. The
results demonstrated that the use of FBs is an effective method to im-
prove airflow distribution and cooling efficiency within DCs, while the
optimal thermal environment appeared when the front doors of racks
were equipped with 20-cm FBs with 75° angle in the vertical direction.
Tatchell-Evans et al. [19] performed an experimental and theoretical
investigation of bypass air in DCs employing cold aisle containments

Fig. 1. Main equipment in a typical data center. Fig. 2. The breakdown of energy consumption in typical DCs.

2
X. Yuan, et al.
and its impact on energy consumption. The results demonstrated that
the airflow bypass rate at typical cold aisle plenum pressures is about
20% increasing the flow rate demand of CRACs. If bypass airflow is
minimized, the total energy consumption can be reduced about 8.8%.
In addition, excessive plenum pressure is not conductive to energy
conservation, which increases energy consumption up to 16%. Ni et al.
[20] used CFD modelling to simulate the thermal distribution and air-
flow conditions in a small DC and investigate the optimum airflow
distribution conditions. The results of CFD simulation showed that in
the optimal conditions high performing airflow distribution is reached.
Wibron et al. [21] used CFD software to compare the performance
metrics of partial aisle containments and raised floor data center. They
found that combining the performance metrics could improve the air-
flow distribution in the raised floor configurations. Huang et al. [22]
also established numerical model to compare and analyse the airflow
distribution in DCs with three different airflow patterns in 2017. The
results showed that compared to under-floor and row-level air supply
pattern, rack level airflow pattern performed the best with highest
cooling efficiency. As a result, the existing researches on the optimi-
sation of airflow organisation focused more on room-level and row-
level parameters optimisation instead of rack-level airflow optimisa-
tion. However, the cause of non-uniform airflow distribution in the DC
is from the servers within racks, thus more researches should focus on
the improvement of cooling efficiency and new method of airflow dis-
tribution within the rack.
The novelty of this paper is to tilt the servers and change 4 different
tilting angles (15°, 30°, 45° and 60°) to change the direction of cold
airflow so that the cooling efficiency and thermal environment can be
improved and the temperature of rack hotspots and the possibility of
potential local hotspots can be reduced in DCs. The objectives of this
paper are to: (i) develop a CFD DC model to Airpak 3.0 packages, (ii)
validate the CFD model based on the real DC in Nanjing, China, (iii)
apply the CFD model to the DC scenarios where the servers are placed
evenly and with different tilted angles, and (iv) with different tilt angles,
optimize power of servers. The organization of this paper is as follows. In
Section 1, the recent development status of DCs and the investigations of
airflow management strategy are summarized, and the innovation of our
airflow optimisation method is carried out. Then, Section 2 introduces
the studied DC and the experimental setup. In Section 3, the simulation
information is described in detail mainly including CFD model, para-
meter settings, grid independent test, simulation conditions, boundary
conditions, etc. In Section 4, the CFD model is validated by the experi-
mental data, and then new CFD models with evenly placed servers with
different inclination angles are established. The simulation results with
different server placement and different racks’ heat load are discussed
and compared, while Section 5 draws the conclusions.
2. Methodology
2.1. Data center description
The parameters setting and equipment of the studied DC is detailed
presented in our previous paper [16]. The DC serves Nanjing Tech
University in Nanjing, China to supply network creation, information
processing and communication support working 24/24, 365 days/year.
This DC has the dimensions of 11.0 m in length, 8.0 m in width and
4.0 m in height. Fig. 3 illustrates the configuration and specifications of
the studied DC. There are two CRACs to generate the cooling air and
restrain the heat generation of the servers, and the air circulation
process within the DC is shown in Fig. 4. The DC is equipped with 2
rows of racks named as rack A and rack B, while each has 9 racks with
different IT load. The parameters setting and involved equipment are
summarised in Table 1 [16,23]. The space between rack A and rack B is
closed, namely closed aisle containment (CAC), while the space be-
tween the racks and the walls are the hot aisles. The air supply tem-
perature to the racks in all tiles are assumed to be the same.
3
Applied Thermal Engineering 164 (2020) 114444
2.2. Experimental setup
In this study, experiments were carried out in July 2017 in the li-
brary DC of Nanjing Tech University, Nanjing, China. The objectives of
the experiments were to obtain the exhaust air temperature distribution
of the racks in an operating DC. According to the server running in-
formation over the whole year operation condition from the FM de-
partment, rack A was selected for the study. Fig. 5 shows the heat load
of each rack in the rack A, and rack A1, A4 and A9 are not in operating
condition. In the experiments, rack A2 – A8 were selected to be the
measured objectives. The location of temperature sensors is shown in
Fig. 6, describing the measurement arrangement and the locations pf
sensors. For each rack, there are installed 6 temperature sensors in the
rear door of the racks. The first temperature sensor is set 200 cm above
the plenum, while the distance between the adjacent sensors is the
same, which is set as 300 cm. The supply air temperatures (SATs) of
rack A2-A8 through the plenum were measured with a hand-held air
flow anemometer. According to the measurement results, the SATs
varied from 22 °C to 22.2 °C, while the supply air temperature of CRACs
was set as 22 °C. Under this condition, the SATs of rack A2-8 were as-
sumed to be the same, which was equal to the CRACs’ SAT of 22 °C.
2.3. Experimental condition and measuring instruments
To improve the reliability and obtain more accurate experimental
results, there were carried out together 9 experiments between 15th
July and 17th July, while each day was divided into three groups of the
morning, afternoon and evening experiments. Each measurement case
was lasted 2 h. Thus, 9 groups of experimental results were obtained
during three days. The results were the average value of all measured
values (average of three days data, except of experimental data with
large deviation).
Type-K thermocouples of nickel-chromium-silicon were used as the
temperature sensor, which were connected to the Agilent 34972A data
acquisition logger. This Agilent data acquisition logger recorded the
exhaust air temperature of each sensors and transferred the data to a
personal computer. The specific parameters of the thermocouples and
handheld air flow anemometer were shown in Table 2. The temperature
data was received by Agilent data acquisition logger every 10 s, which
meant there were approximately 720 temperature data for each test
point after 2-h measurement time. Thus, the temperatures of each test
point during the 2-h measurement time were the arithmetic mean of
720 data avoiding the interference of transient temperature fluctuation
on the actual exhaust air temperature distribution.
Fig. 3. Configuration and specifications of the studied DC (Top view).
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Fig. 4. The air flow distribution within the studied DC.

Table 1 3. Numerical simulation

DC and equipment parameters [16,23].
Items Description Value
3.1. Data center model

DC room A commercial CFD software Airpak3.0 is used to establish the nu-

Dimension of CACs
Number of perforated tiles
Raised floor height
Dimension of perforated tiles
Porosity of perforated tiles
CRACs
Number of CRACs
Dimension of CRACs
Flow air velocity of each CRAC
Supply air temperature (SAT)
Supply air cooling capacity
Distance between CACs and CRACs
Area of each air vent of CRACs
Racks
Number
Dimension of each rack
Door porosity
5.4 m × 1.8 m × 2.2 m (H) merical DC model. Fig. 7 shows the numerical model, which is based on
Total of 27 in three rows
the actual configuration and specification of the DC shown in Fig. 3.
0.45 m
0.6 m × 0.6 m The locations and heat load of the servers in the numerical model were
45% in according with the actual data center. The air conditioning system
for air supply applies under-floor air supply mode, while the complete
2 air circulation process is as follows: (1) 22 °C cool air is generated in
1.8 m × 0.8 m × 2.25 m (H) CRACs and is transported with air supply speed of 5.33 m/s via the
5.33 m/s underfloor plenum; (2) the cool air rises through the floor vent and
22 °C
65.8 kW
enters into the CAC between rack A and rack B; (3) the cool air is drawn
1.6 m via the servers and heated by the IT loads of the servers; (4) Finally, the
0.45 m2 hot air enters the hot aisles behind the two rows of racks and returns to
the CRACs. The air supply amount of CRACs is equal to the total airflow
Total of 18 in 2 rows required by the server fans.
1.2 m × 0.6 m × 2.2 m (H)
65%

3.2. Governing equation

The main research object was the exhaust air temperature of racks;
thus both the steady-state and unsteady-state flow fields were needed
be calculated in the simulation process. According to Ni et al [20] and
Alkharabsheh et al. [24], compared to k-ω turbulent model, Shear
Stress Transport (SST) and Reynolds Stress Model (RSM), the k-ε tur-
bulence model performed better in terms of overall operation perfor-
mance, accuracy, simulation time and coincidence degree with ex-
periments. The simulation model is selected as described in our
previous paper [16]. Wibron et al. [25] recently examined the perfor-
mance of k-ε turbulence model, the Reynolds Stress Model (RSM) and
Detached Eddy Simulations (DES) in CFD modelling of data centers.
They concluded that RSM turbulent model has higher precision than k-ε
turbulence model and spent fewer solution times than DES turbulent
model. However, up to now, the standard k-ε turbulence model has
been adopted widely and performed well [13,24,26–29]. Standard k-ε
turbulence model is adopted with an enhanced wall function, followed
by a finite-volume approximation of the Reynolds-averaged Naiver-
Stokes (RANS) [26–27,30].
Fig. 5. Heat load of each rack in rack A9. Some assumptions were set as described in our previous works

4
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Fig. 6. Location of temperature sensors in the experiments.

Table 2 Ì¿
(u ) + u · u = p+ ·(T ) + g
Specific parameters of thermocouples and air flow anemometer. t (2)
Items Type Measuring range Uncertainty
(h ) + ·(h u ) = ·[keff T ] + S
Thermocouple Type-K 0–400 °C ± 0.4% t (3)
Handheld air flow KIMO Temperature: ± 0.4%
Anemometer VT200/ −20 °C to 80 °C The Stress tensor T is given by
FC300 Velocity: 0.15–30 m/s ± 2.0% Ì¿
T = veff · u (4)

[16,23]. The governing equations (Continuity, Momentum and Energy

conservation equation) are given as follows [16,23,31]. 3.3. Boundary conditions

·u = 0 (1)
The boundary conditions are the same as our previous paper [16].

5
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Table 4
Parameter settings for boundary conditions [16].
Boundary conditions Symbol Equation Value

Inlet airflow velocity U0 – 5.33 m s−1

Tile flow rate Qt Qt = U0Atile 1.92 m3 s−1
Inlet air temperature T0 – 22 °C
Server flow rate Qs Qs = Qt/4 0.48 m3 s−1
Server heat generation Ps Ps = ρ cp Qs ΔT 5040 w
Perforated tiles porosity σt – 45%
Rack porosity σr – 65%

same, which is set as 90 mm. In addition, the first server is 90 mm above

Fig. 7. Simulation model of the applied DC.
The specifications of simulation cases are shown in Table 3, while the
parameters for the boundary conditions are shown in Table 4. The
convergence criteria were set as follows: the residuals for both the ve-
locity in the directions of X, Y, and Z and the continuity were 10−3, and
that for energy was set at 10−6 [24,32]. The simulation converged in
around 1 h in the scenario 1, while the convergence time of the simu-
lation in the scenario 2 and 3 is about 2 h. Enhanced wall treatment is
applied to solve near-wall region problem.
3.4. Simulation conditions
The numerical simulation was divided into three scenarios. In the
scenario 1, all the parameters and specification were set according to
the actual DC. After getting the simulation results, the numerical model
is validated by the experimental results. In the scenario 2, once the
numerical model and actual model was validated to be consistent, the
numerical model was made some changes in servers’ tilted angles and
heat load, but kept other DC’s parameters and specification unchanged.
To obtain general results instead of the results from special working
conditioning, 8 servers were placed evenly for each rack. In avoidance
of too many grid numbers causing a very long convergence time, 2
racks (rack A7&A8) were selected for the detailed analysis. In this
condition, the convergence time is still more than 2 h. The heat load of
each server is set to be the same 1000 W, which can be considered as
high-density racks. The distance between adjacent servers is also the
the plenum. In the scenario 2, tilted servers were applied with 4 dif-
ferent tilting angles (15°, 30°, 45° and 60°) and the effect of tilted ser-
vers on the improvement of thermal environment and cooling efficiency
were studied. As a result, the optimal tilted angle recorded as X° is
obtained as well as the optimal exhaust temperature distribution. Based
on the scenario 2, different IT load were introduced in the scenario 3
and the exhaust temperature distribution as well as energy saving rate
between the model with horizontal servers and the servers with X° in-
clination angle was compared. All the parameters setting of each sce-
nario are shown in Table 5. Fig. 8 shows the sketch map of servers’
position in one rack with different tilted angles.
3.5. Grid generation
According to Ni et al. [20], grid generation plays an important role
in the calculation accuracy of the simulation results. Coarse grid and
round-off errors caused by tremendous increasing of mesh number can
increase the chance of calculation errors, while too few mesh may lead
to the inaccuracy of simulation [16]. Thus, a grid proper quality and
number should be generated in this DC model to obtain accurate si-
mulation results. In this paper, there are three scenarios and altogether
18 different simulation cases, and the refined meshes were all applied
near the servers and vents. In all 18 simulation cases, the vents are set
as the same, but the servers are divided into 2 groups, which are sce-
nario 1 and scenario 2 and 3. Thus 2 different grid independence tests
should be done. Hexa unstructured mesh is applied in the model con-
struction process, which generates non-uniform grid to minimize the
mesh number efficiently but it guarantees the same calculation accu-
racy. The grid independence test method mentioned in our previous
work [16], which is also used in the papers of Siriwardana et al. [11]
and Nada et al. [33], is applied in this research.
As shown in Table 6, there are two sets of mesh numbers applied in
the numerical simulations, which are for the scenario 1 and the scenario
2 and 3. Figs. 9 and 10 shows the mesh sensitivity analysis of object

Table 3
Simulation model’s specifications and boundary conditions [23].
Objects Type Dimension/cm × cm (×cm) Quantity Remark

Walls Adiabatic 1100 cm × 400 cm 2 External walls

800 cm × 400 cm 2
Floor Adiabatic 1100 cm × 800 cm 1 External walls
Ceiling Adiabatic 1100 cm × 800 cm 1 External walls
Plenum Adiabatic 1100 cm × 800 cm 1 Exclude CAC
Top of CAC Adiabatic 540 cm × 180 cm 1
Perforated tiles Vent 60 cm × 60 cm 27 45% free air ratio
Front door of racks Vent 220 cm × 60 cm 18 65% free air ratio
Rear door of racks Vent 220 cm × 60 cm 18 65% free air ratio
CRACs Block 180 cm × 85 cm × 180 cm 1
Fan 180 cm × 15 cm 1 Air outlet
180 cm × 85 cm 1 Air inlet
Servers Partition 80 cm × 46 cm × 2 12 Upper and lower
80 cm × 9 cm × 2 12 Sides
46 cm × 8.5 cm 12 Front
Source 79 cm × 45 cm × 4 12 Uniform layout

6
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Table 5 respectively. Thus, 574,938 and 1,098,659 are selected as the mesh
Parameters setting of each scenario. number in the simulations for the scenario 1 and the scenario 2 and 3,
Scenario Heat load per rack (kW) Tilted angle (°) respectively. It should be emphasised that Fig. 9 is quoted from our
previous research [16,23]. According to CFD Airpak3.0, the mesh
Scenario 1 8 0 quality can be checked after grid generation, while the quality of 99%
Scenario 2 8 Case I 0
meshes can meet the software requirement.
Case II 15
Case III 30
Case IV 45
Case V 60 4. Results and discussions
Scenario 3 Case1 8 0 X
Case2 12
The simulation is divided into 3 scenarios. In scenario 1, the si-
Case3 16
Case4 20 mulation model is validated with the experimental data. In scenario 2,
Case5 24 some changes are made in the servers’ heat load and tilting angles. Rack
Case6 28 A7 and A8 were selected as the studied racks, where each equipped
Case7 32
with 8 one-kW servers placed evenly. Then, the optimum angle and
exhaust temperature distribution can be obtained by adjusting the
horizontal angle (0°, 15°, 30°, 45°, 60°) of servers. Under this circum-
maximum temperatures with grid number in the scenario 1 and the
stance, the optimal angle is record as X°. Based on scenario 2, scenario 3
scenario 2 and 3, respectively. The object maximum temperature
studied the exhaust air distribution optimisation and energy saving
reaches the minimum and is stable when the mesh numbers are no less
potential between the model with 0° servers and X° servers under 7
than 574,938 and 1,098,659 for the scenario 1 and the scenario 2 and 3,
different heat load levels of server.

Fig. 8. Side view of servers’ position in one rack with different angles.

7
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Table 6
The different mesh numbers for the scenario 1 and the scenario 2 and 3 applied in the simulation.
Category Mesh numbers

Scenario 1 168,022 179,774 222,223 330,050 574,938 919,342 1,687,284

Scenario 2 and 3 274,534 549,983 823,776 1,098,659 1,372,632 1,647,901 1,921,826

In the simulation part, the cloud map of exhaust air temperature is

Fig. 9. Mesh sensitivity analysis of object maximum temperatures variations
with grid number in the scenario 1 [16].
shown in Fig. 12. There is heat accumulation in the top part of rack 6,
while heats are accumulated in the lower part of racks 2, 3, 5 and 7–8.
In addition, the exhaust air temperature of rack 4 is uniform because of
no servers placed within the rack. Thus, the exhaust air temperature
distribution of 6 racks is un-uniform. By means of analysing the exhaust
temperature map, the values of the points in the cloud map corre-
sponding well with the test points in the experiment are recorded and
sorted. The temperature values of each record points in the simulation
part are also presented in Fig. 11 as the dotted lines.
The temperature values of the rack A2-A8 were validated between
the measurements and simulation results in Fig. 11. The simulated
temperatures of each outlet measuring points were slightly higher than
those in the experimental results except in the rack A4. There are two
reasons that may lead to higher exhaust air temperature than in si-
mulations: one is that the heat power of the servers in the simulation
are set according to their full load condition, while the actual servers
may not operate with the full load. Another reason could be that the
internal structure of the servers modelled is not exactly the same than
actual one. In the rack A4, the simulation results are slightly lower than
experimental results. The reason for this temperature difference is that
there is no server placed in real rack A4. Thus, the exhaust air tem-
perature of rack A4 is equal to the CRACs’ supply air temperature, while
the CRACS’ real supply air temperature is slightly higher than the set
temperature in the DC. According to Fig. 11, it is obvious that the
biggest temperature difference between simulation and experimental
results exists in rack A8. The relative error of 6 points in the rack A8
between experimental and simulation results are presented in Fig. 13.
The maximum relative error is around 4%. Thus, the exhaust air tem-
perature distribution achieved with simulations fits well with the
measurement data.

4.2. Simulation results in the scenario 2

In the scenario 2, the total heat load for each rack is 8 kW. The
simulation results in the scenario 2 are shown in Fig. 14. It could be
noted that there are obvious heat accumulations at the bottom of both
rack A7 and A8 in Case I. The highest temperature of rack A7 and A8 is
Fig. 10. Mesh sensitivity analysis of object maximum temperatures variations 28.0 °C and 28.4 °C, respectively. According to our previous work [16],
with grid number in scenario 2 and 3. rack hotspots were defined as the highest exhaust air temperature
points in each rack, which were the most possible locations that could
4.1. Statistical analysis and validation study damage the servers once the temperature of rack hotspot is higher than
45 °C and become local hotspot [34]. According to ASHRAE Technical
In the experiment part, after data sorting of 9 exhaust temperature Committee (TC) 9.9 [34], the existence of local hotspots leads to non-
data sets, 7 groups of them were saved for the arithmetic values of each uniform air distribution, which would shorten operational life span of
test point. The measurement data from the morning of 16th July and servers and increase the energy use in DCs. Thus, although the highest
the evening of 17th July where neglected because of a big difference
from other sets of data. For the rest 7 groups of experimental data, Table 7
standard deviation of them in each measuring points of Racks A2-8 was Sensitivity analysis of 7 groups of temperature data in each measuring points of
done. As shown in Table 7, the standard deviation of temperature Racks A2-8.
measuring point 1 in Rack A7 was the largest of 0.33, while those of the Standard deviation A2 A3 A4 A5 A6 A7 A8
rest measuring points were no more than 0.2. All the standard devia-
1 0.09 0.15 0.05 0.08 0.08 0.33 0.10
tions of 42 measuring points were small and acceptable. Thus, the ar-
2 0.05 0.12 0.05 0.12 0.16 0.20 0.10
ithmetic mean of these 7 groups of exhaust temperature data sets can be 3 0.00 0.00 0.05 0.05 0.14 0.10 0.08
used as the actual temperature values in each measuring points. The 4 0.14 0.05 0.05 0.05 0.13 0.15 0.08
arithmetic values of each test point are presented in Fig. 11 as the solid 5 0.13 0.09 0.05 0.04 0.14 0.08 0.05
lines. 6 0.05 0.04 0.10 0.05 0.08 0.08 0.06

8
X. Yuan, et al.
Fig. 11. Comparison of temperature value of six outlet measuring points in rack
A2 - A8 between experimental (solid line) and simulation (dotted line) results.
temperature of rack A7&A8 could not reach the level of the local hot-
spot, the risk of server damage should be prevented. Moreover, the
exhaust air distribution of rack A7 and A8 were still non-uniform. A
more uniform air distribution can decrease the energy consumption
within DCs.
As shown in Fig. 14, the maximum exhaust air temperature of rack
A7 and A8 are 28.0 °C and 28.4 °C in Case I. The exhaust air tempera-
ture decreased to the lowest points of 26.7 °C and 26.8 °C in Case III,
respectively. There is an increase of the maximum air temperature of
29.4 °C and 29.6 °C, in Case Ⅴ. In addition, the heat accumulation of
Case Ⅳ and Case Ⅴ are different to those in Case I-III.
The reasons for the abnormal heat accumulation are can be depicted
in Fig. 15. Due to the similar server angle, the exhaust air distribution of
Case II and Case III are similar, while the exhaust air distribution of
Case Ⅳ and Case Ⅴ are similar too. Fig. 15 just shows the velocity vector
of rack A8 in Case I, Case III and Case Ⅴ. In Case I, the cool air from the
vents inclines upward into rack A8 and a part of the cool air is blocked
by the servers, while the rest of the cool air flows through the tunnel
between the servers in a horizontal direction. A small part of the cold
air flowing through the channel take away the heat from the terminal of
the servers, while the rest become bypass cold air and flow out of the
rack. Under this circumstance, the heat removal by the cool air is very
limited in Case I. The heat accumulation zone always exists in the
Fig. 12. The simulation results of exhaust air temperature distribution of rack A2-A8.
9
Applied Thermal Engineering 164 (2020) 114444
Fig. 13. The relative error of 6 test points in rack A8 between experimental and
simulation results.
terminal of servers which is shown in Fig. 15. In addition, as shown in
Fig. 14 of Case I, a high percentage of bypassing air flows directly from
the upper side of the rack where no server is placed. Thus, the cool air
supplied to the bottom of the rack is limited, causing the heat accu-
mulated at the bottom of the rack.
However, the application of 30° tilted servers in Case III keep the
tilted angle of servers and the inlet cool air direction of the rack basi-
cally the same. Moreover, the occupied spaces of inclined servers inside
the rack are larger than servers of Case I to reduce the bypassing effect.
It can be seen in Fig. 15 of Case III that although a part of cool air is still
blocked by the servers, the cool air flows along the upper and lower
sides of the servers, and then the cool air inclines downward and dis-
sipates the heat at the terminal of the servers. The most direct effect of
the application of 30° tilted servers is shown in Fig. 14 of Case III, where
the temperature of rack hotspot of rack A8 decreases from 28.4 °C in
Case I to 26.7 °C in Case III, and the heat accumulation of rack A8 is
reduced and the highest temperature rises to the middle of the rack.
In Fig. 15 of Case Ⅴ, when the tilted angle of servers continues rising
to 60° in Case Ⅴ, the distance between neighbouring servers decreases
greatly and almost stick together, which blocks the flow pattern of cool
air extremely. Considering that the height of rack cannot be changed,
the distance between adjacent tilt servers cannot be increased. The cool
air rises up to the top and cross the top server, which can be regarded as
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Fig. 14. The cloud map of exhaust air temperature of rack A7 and A8 in the scenario 2 with different server tilted angles.

Fig. 15. The air flow direction of Case I, Case III and Case.

significant airflow bypass. Fig. 16 shows the velocity profiles of Case Ⅴ

in the scenario 2. In Fig. 16, the majority of cool air flows along the
inclined downward direction out of the rack, while very small amount
of air flows to the terminal of racks. The heat generated from rack A7
and A8 in the case Ⅴ is accumulated in the middle of rear door in the
horizontal direction. However, in fact, the heat accumulated in the
terminal of servers spread to the left and right sides of the servers, as
there is little cool air supplied across the server. In this case, the in-
clined downward exhaust air (seen from Fig. 16) isolates the accumu-
lated heat in the terminal of servers to flow out to the rear door of racks.
Therefore, as shown in Fig. 14 of Case Ⅴ, the heat is accumulated in the
interface of two racks.
The mean exhaust air temperature of rack can be obtained by the
Airpak 3.0 software. Fig. 17 further shows the comparison of exhaust
air distribution in Case I-Ⅴ, concerning the minimum-maximum range
and the mean temperature of rack A7 and A8 in the scenario 2. The
blue1 lines in Fig. 17 represent the mean temperature of 120 simulated
points in rack A7 and A8. The mean temperature of rack A8 is 25.7 °C in
Case I and decreases modestly to the lowest point of 25.5 °C in both
Case II and Case III, where there was an exponential increase to highest

1
For interpretation of color in Fig. 17, the reader is referred to the web
Fig. 16. The velocity magnitude sketch of Case Ⅴ in scenario 2. version of this article.

10
X. Yuan, et al.
Fig. 17. Minimum-maximum range and mean temperature of rack A7 and A8.
value of 27.1 °C in Case Ⅴ. The lowest mean temperature of Rack A8
appears in Case II and Case III. The mean temperature in rack A7 is the
same as that in rack A8, whose lowest mean temperature appears in
Case II and Case III. Under this circumstance, as the mean temperatures
of Case II and Case III are the same, the temperature range of rack A7
and A8 in Case II and Case III are compared in Fig. 17. The temperature
ranges of rack A8 in Case II and Case III are 4.7 °C and 4.2 °C, respec-
tively, while the temperature ranges of rack A7 in Case II and Case III
are 4.2 °C and 3.8 °C, respectively. The lower the range, the more uni-
form the temperature distribution. Considering the temperature uni-
formity inside the rack, the exhaust air temperature distribution in Case
III with the tilted of 30° is the most preferred in both rack A7 and A8.
As a summary, combined the results from Figs. 14 and 17, 30° is
regarded to be the most optimal tilted angle. The exhaust air tem-
perature distributions of both rack A7&A8 are the best with the servers’
angle of 30 °C. Therefore, in this server distribution, the heat accumu-
lation inside the rack can be significantly prevented.
4.3. Simulation results in the scenario 3
According to scenario 2, the best air temperature distribution is
achieved at the application of 30° tilted servers. In scenario 3, the heat
load of each rack is changed from 8 kW to 32 kW shown in Case 1-7. In
each case, exhaust mean temperature drop of rack A8 before and after
applying 30° tilted servers is compared as well as the temperature drop
of maximum temperature before and after using 30° tilted server. In this
scenario, the mean exhaust air temperature of rack can also be obtained
by the Airpak 3.0 software
As shown in Fig. 18, exhaust mean temperature reduction of rack A8
with 0° and 30° tilted servers steadily increases with the increase of heat
load, until the maximum value is about 1.44 °C in Case 7. In addition,
the temperature reduction of the maximum temperature before and
after applying 30° tilted servers in case 1-7 increases with the increases
of heat load. In case 7, the temperature reduction of the maximum
temperature before and after applying 30° tilted servers reaches almost
3.3 °C, which successfully lowers the maximum temperature of rack A8
from around 45.3 °C to 42 °C. According to ASHRAE Technical Com-
mittee (TC) 9.9 [34], the operating temperature of the servers should be
lower than 45 °C, or it will threaten the normal operation of servers and
malfunction [33]. Thus, as shown in Table 8, the application of 30°
tilted servers limits the maximum temperature of rack A8 below 45 °C
in case 7, while the maximum temperature of rack A8 in case 1-6 de-
creases to varying degrees. Thus, the application of 30° tilted servers
can improve the airflow distribution and cooling effects.
Table 9 shows the heat transfer from the rack with 0° and 30° tilted
servers in case 1-7. According to Eq. (5), the heat removal is calculated
and presented in Table 9. The specific heat capacity of air (cp) is about
11
Applied Thermal Engineering 164 (2020) 114444
Fig. 18. The mean and maximum temperature reduction of rack A8 with 0° and
30° tilted servers.
1 kJ/(kg K), while the air density ( ) is around 1.29 kg/m3.
Q = c· m · t (5)
m = ·v (6)
t 0 = t 0° t 30° (7)
t1 = tex ts (8)
Q0 cp· m · t 0 t0
r= = =
Q1 cp· m · t1 t1 (9)
As listed in Table 9, for one rack, the heat taken away before and
after applying 30° tilted servers increases steadily with the increase of
heat load in case 1-7, which finally reaches 1611 kJ per hour in case 7.
Compared to the heat taken away in the rack with horizontal servers,
under the same conditions, at least 5.7% extra heat is taken away in the
rack applying 30° tilted servers with 7 different heat loads. In addition,
the extra heat taken away reaches 10% when the heat load reaches
32 kW in each rack.
5. Conclusion
An optimization method using inclined servers for the temperature
distribution inside the racks of the data center was proposed. The si-
mulation model validated with the on-site experimental results were
applied to study the effect of 4 different server tilted angles (15°, 30°,
45° and 60°) on the optimum operating condition. The contributions of
this study are as follows:
1. The airflow profiles and cooling efficiency are both improved with
the server inclined angel of 15° and 30°, while the more uniform
thermal environment is achieved when the angle of servers is set to
be as 30° in horizontal direction.
2. The maximum and mean exhaust air temperature of rack is reduced
to varying degrees with the application of 30° tilted servers.
Especially when the heat load of rack is 32 kW, the maximum ex-
haust air temperature is significantly reduced from the operation
risk temperature of 45.3 °C to accepted temperature level of 42 °C
(ASHRAE) with the application of 30° tilted servers
3. Compared to the heat removal effect of the horizontal and tilted
servers, under the same conditions, at least 5.7% extra heat is re-
moved in the rack applying 30° tilted servers. In addition, the extra
heat removal percentage reaches to 10% when the heat load is
32 kW in each rack.
X. Yuan, et al. Applied Thermal Engineering 164 (2020) 114444

Table 8 liquid-cooled data centers: from chip to the cooling system, Appl. Energy 205
The temperature parameters between server inclination angle of 0° and 30° in (2017) 1165–1188.
scenario 3. [9] S.V. Patankar, Airflow and cooling in a data center, J. Heat Transfer 132 (7) (2010)
073001.
Case 1 2 3 4 5 6 7 [10] Y. Joshi, P. Kumar, Energy Efficient Thermal Management of Data Centers, Springer
Science & Business Media, 2012.
Heat load of each rack (kW) 8 12 160 20 24 28 32 [11] J. Siriwardana, S.K. Halgamuge, T. Scherer, W. Schott, Minimizing the thermal
Mean temperature drop (K) 0.21 0.35 0.48 0.64 0.82 1.11 1.44 impact of computing equipment upgrades in data centers, Energy Build. 50 (2012)
81–92.
Maximum temperature of rack A8 28.4 31.4 34.2 36.9 39.9 42 45.3
[12] T. Ding, Z.G. He, T. Hao, Z. Li, Application of separated heat pipe system in data
with 0° servers (°C)
center cooling, Appl. Therm. Eng. 109 (2016) 207–216.
Maximum temperature of rack A8 26.8 29.1 32.2 34.3 37.4 38.8 42.0
[13] A. Almoli, A. Thompson, N. Kapur, J. Summer, H. Thompson, G. Hannah,
with 30° servers (°C) Computational fluid dynamic investigation of liquid rack cooling in data centers,
Appl. Energy 89 (2012) 150–155.
[14] H. Zhang, S. Shao, H. Xu, H. Zou, C. Tian, Free cooling of data centers: a review,
Table 9 Renew. Sustain. Energy Rev. 35 (2014) 171–182.
[15] R. Cushing, L. Doherty, Next Generation Data Centers, MSC Business Development
The heat removal capacity with 0° and 30° tilted servers in case 1-7. LTD., 2009.
Case 1 2 3 4 5 6 7 [16] X. Yuan, Y. Wang, J. Liu, X. Xu, X. Yuan, Experimental and numerical study of
airflow distribution optimisation in high-density data center with flexible baffles,
Build. Environ. 14 (2018) 128–139.
Cp [kJ/(kg K)] 1
[17] J. Cho, J. Yang, W. Park, Evaluation of air distribution system's airflow performance
m (kg/h) 1.29 kg/m3·0.24 * 3600 m3/h = 1114.6 kg/h
for cooling energy savings in high-density data centers, Energy Build. 68 (2014)
Δt1 (K) 3.5 5.4 7.6 9.2 11 12.6 14.4
270–279.
Δt0 (K) 0.2 0.35 0.5 0.7 0.85 1.1 1.5 [18] T. Lu, X. Lü, M. Remes, M. Viljanen, Investigation of air management and energy
Q0 (kJ/h) 223 392 559 783 950 1118 1678 performance in a data center in Finland: case study, Energy Build. 43 (2011)
Q1 (kJ/h) 3900 6018 8470 10,253 12,260 14,043 16,049 3360–3372.
r (%) 5.7 6.5 6.6 7.6 7.7 8.0 10 [19] M. Tatchell-Evans, N. Kapur, J. Summers, H. Thompson, D. Oldham, An experi-
mental and theoretical investigation of the extent of bypass air within data centers
employing aisle containment, and its impact on power consumption, Appl. Energy
186 (2017) 457–469.
Acknowledgement [20] J. Ni, B. Jin, B. Zhang, X. Wang, Simulation of thermal distribution and airflow for
efficient energy consumption in a small data centers, Sustainability 9 (2017) 664,
This work is supported by the National Natural Science Foundation https://doi.org/10.3390/su9040664.
[21] E. Wibron, A. Ljung, T.S. Lundström, Comparing performance metrics of partial
of China (Grant No. 51806096) and Natural Science Foundation
aisle containments in hard floor and raised floor data centers using CFD, Energies
of the Jiangsu Higher Education Institutions of China (Grant No. 12 (2019) 1473, https://doi.org/10.3390/en12081473.
18KJB560007). We acknowledge the help from Information Center of [22] Z. Huang, K. Dong, Q. Sun, L. Su, T. Liu, Numerical simulation and comparative
analysis of different airflow distributions in data centers, Proc. Eng. 205 (2017)
Nanjing Tech University for providing on-site experimental sites.
2378–2385.
[23] X. Yuan, X. Zhou, J. Liu, Y. Wang, R. Kosonen, X. Xu, Experimental and numerical
Appendix A. Supplementary material investigation of an airflow management system in data center with lower-side
terminal baffles for servers, Build. Environ. 155 (2019) 308–319.
[24] S. Alkharabsheh, J. Fernandes, B. Gebrehiwot, D. Agonafer, K. Ghose, A. Ortega,
Supplementary data to this article can be found online at https:// Y. Joshi, B. Sammakia, A brief overview of recent developments in thermal man-
doi.org/10.1016/j.applthermaleng.2019.114444. agement in data center, J. Electron. Packag. 137 (2015) 1–19.
[25] E. Wibron, A.L. Ljung, T.S. Lundström, Computational fluid dynamics modelling
and validating experiments of airflow in a data center, Energies 11 (2018) 644,
References https://doi.org/10.3390/en11030644.
[26] Z. Song, Thermal performance of a contained data center with fan-assisted per-
[1] B. Durand-Estebe, C.L. Bot, J.N. Mancos, E. Arquis, Simulation of a temperature forations, Appl. Therm. Eng. 102 (2016) 1175–1184.
adaptive control strategy for an IWSE economizer in a data center, Appl. Energy [27] S.W. Ham, J.W. Jeong, Impact of aisle containment on energy performance of a data
134 (2014) 45–56. center when using an integrated water-side economiser, Appl. Therm. Eng. 105
[2] V. Depoorter, E. Oro, J. Salom, The location as an energy efficiency and renewable (2016) 372–384.
energy supply measure for data centers in Europe, Appl. Energy 140 (2015) [28] E. Cruz, Y. Joshi, Coupled inviscid-viscous solution method for bounded domains:
338–349. application to data-center thermal management, Int. J. Heat Mass Transf. 85 (2015)
[3] K. Ebrahimi, G.F. Jones, A.S. Fleischer, A review of data center cooling technology, 181–194.
operating conditions and the corresponding low-grade waste heat recovery op- [29] L. Phan, C. Lin, A multi-zone building energy simulation of a data center model with
portunities, Renew. Sustain. Energy Rev. 31 (2014) 622–638. hot and cold aisles, Energy Build. 77 (2014) 364–376.
[4] S. Sevencan, G. Lindbergh, C. Lagergren, P. Alvfors, Economic feasibility study of a [30] ANSYS, ANSYS Fluent User’s Guide, Release 15, ANSYS, Inc., Canonsburg, PA,
fuel cell-based combined cooling, heating and power system for a data center, 2013.
Energy Build. 111 (2016) 218–223. [31] Fluent Inc, Airpak 3.0 User’s Guide, Fluent Inc, Lebanon, 2007.
[5] J.G. Koomey, Growth in Data Center Electricity Use 2005 to 2010, Analytics Press, [32] K. Zhang, X. Zhang, S. Li, G. Wang, Numerical study on the thermal environment of
Oakland, CA, 2011. UFAD system with solar chimney for the data center, Energy Proc. 48 (2014)
[6] R. Brown, et al., Report to Congress on Server and Data Center Efficiency: Public 1047–1054.
Law 109-431, Lawrennce Berkeley National Laboratory, 2007, https://doi.org/10. [33] S. Nada, H. El-Batsh, H. Elattar, N. Ali, CFD investigation of airflow pattern, tem-
2172/929724. perature distribution and thermal comfort of UFAD system for theatre buildings
[7] J. Whitney, P. Delforge, Data Center Efficiency Assessment, Natural Resources applications, J. Build. Eng. 6 (2016) 274–300.
Defense Council, New York, 2014. [34] ASHRAE, Thermal guideline for liquid cooled data processing environments, White
[8] A.H. Khalaj, S.K. Halgamuge, A review of efficient thermal management of air-and paper prepared by ASHRAE Technical Committee (TC) 9.9, 2015.

12