IT Data Center QA & Specification

Guideline

IT DATA CENTER QA & SPECIFICATION

Objective: To prepare a complete & structured guideline for IT datacenter which will also provide standard working reference on planning , implementation & operation stage.

Area of Influence under the Guideline

Power Cooling AFFS Labeling Cabling Drawing & Document Asset Tracking House Keeping ITDC site preparation Cabinet Installation Checklist

Substation, Generator, BBT, MDB, COS, Modular distribution, light & Socket DC power Rectifier/Inverter, UPS, Battery Grounding & Surge protection AC power Server room, Substation & Generator room Equipment & Cable Labeling standard Signal & Optical cable management system Power & Grounding cable management system

Server room preparation, Raise floor, ROXTEC, EQ frame

QAT, Expansion/ Integration, Preventive Maintenance

Highlights of Cooling Guideline

Reference:
TIA: Telecommunications Industry Association

Guidelines Considers New Data Center Existing Data center Improving Cooling Efficiency Solution for High dense Server

ETSI: European Telecommunications Standards Institute

ASME: American society of mechanical engineers Mobile Operator: DTAC, Thailand

Products:
Emerson Network Power American Power Conversion (Schneider Electric) SUN Microsystems (Oracle Corporation)

Environment

Operating Range @ Equipment

Recommended Optimal Range

Overall Ambient Temperature set point @ Air-con Unit

5 to 35 C

21 to 23 C Recommendation 22 C

In Take Temperature of Server (under raise floor)

5 to 35 C

Preferable 20 C

Cold Aisle Temperature

5 to 35 C

< 27 C , Preferable 25 C

Relative Humidity

20% to 80%

40% - 60%

Recommendation for new location:

Cooling Unit

if necessary then dehumidification feature only. Note: also applicable for old air con replacement

CAC (down flow)

During the planning phase of equipments layout, we have to consider : Row Oriented Cooling (Hot-aisle/Cold-aisle Protocol) Placement of cabinet according to air inlet/outlet Spread-out the high-density racks

Gap

Hot aisle (ft)

Cold aisle (ft)

Cold-Cold aisle (ft) 12 ft

Standard Current practice Recommenda tion

Min 2 ft

Min 3 ft

3 ft

(Mandatory and Fixed)

4 ft

14 ft

For smooth O & M, it is necessary to extend the maintenance window . So current practice is followed here.

Fig: The air inlet and outlet characteristics of cabinet

Spread out the high dense server

Fig: Spread Concentrate out the high-density high-density racks racks Fig: high-dense row Fig: Concentrate Spread out the high-dense row

If Aircon unit placement at Cold aisle:

Hot air from the hot aisles crosses over the racks into the cold aisle when returning to the ACs, causing a mixing of hot and cold air that increases the temperature of supply air to the rack fronts. Recirculated air a mixing of hot and cold air will increase at AC end. Align AC units with the air return path (hot aisles).

Misplacement of perforated Recommendation : Perforated tiles impacts Tiles placement :

Recirculated Always at cold air aisle Never Bypass at air vacant flow place Minimum Negative pressure distance ( flow 1.5 meter) from air con

AC

AC

AC

Floor vents placed too close to AC units produce negative pressure due to under floor high air velocity, causing air from the room to be drawn back under the floor

Recommendation: Unused vertical space in rack enclosures causes the hot exhaust from equipment to take a shortcut unused vertical back to the space equipments in rack intake. must be enclosed through filler. Also avoid the unorganized cable management Organized of cable the cabinet of thewhich cabinet may hamper as per the cooled guideline and air product flow. manual

Fig: Organized cable will not hampered air flow Fig: Usages of filler

To measure the ambient temperature and humidity of the data center, may follow these guidelines:

Place sensors in cold aisles, where cold air outlets provide conditioned air to the systems.
Hot-aisle measurements will be typically out the systems recommended operating ranges and therefore are not a valid indication of the required temperature and humidity levels. Place sensors 10 to 15 feet (3 to 4.5 m) apart from AC or in front of every fourth rack in the aisle. Place sensors in the middle of the aisle, between the rows of racks. Place sensors 4.9 feet (1.5 m) above the tiled floor.

Containment of cold aisle:

Using flexible clear plastic barriers to seal the space of tops of the rack, can greatly improve hot aisle/cold aisle isolation. To eliminate short-circuiting (the mixing of hot and cold air) and hot spot. These changes should reduce fan energy requirements and could result in an energy savings on the cooling unit side (15% ~20% ).

Energy Saving Methodology

Energy Saving 15% ~20%

Reference: Emerson Network Power Pacific Gas And Electric Company

Hot spot is created due to: Concentration of high heat dissipating nodes in a same location Huge amount of unorganized under raise floor cables, which prevents proper aeration. Inadequate cooling system.

Solution: Containment solution

Air Distribution Unit. In-Row Air-Handler Unit

Option 1: Air Distribution Unit

Option 2: In Row Air Handler Unit

Cooling loads within racks can be improved by the retrofitting of fan-assisted devices that improve airflow and can increase cooling capacity per rack.

Options:
High-Density Pods in a Low-Density Data Center Autonomous Cabinet System

Fig: High-Density Pods in a Low-Density Data Center

Autonomous Cabinet System : The ACS is a complete data center in a box, incorporating cooling, power, fire and security systems required for very high-density loads. The cooling unit is situated within the enclosure, thus ensuring the maximum effectiveness of cool air delivery to the rack-based equipment.

Redundancy is supportive in cooling systems to permit maintenance of live systems and to ensure the survival of the data center mission if an air conditioning device fails:

Recommendation:

For higher density server, which is also highly business sensitivity, the redundancy approach is sustainable.

Our recommendation at least one redundant Row-oriented provides redundancy at the row level. cooling architecture unit at row level where,
This requires an additional or N+1 AC unit for that particular high dense & cooling sensitive row ( 12 kW Business severity so high Server). High restart time due to failure of cooling High power consumption (> 12 kW) Row area @ 80% occupied

4 Ton AC

4 Ton AC

4 Ton AC

4 Ton AC

5 kW 3 kW

3 kW 4 kW

5 kW 3 kW

3 kW 4 kW

Hot

4 kW
3 kW 4 kW

Cold

3 kW
5 kW 4 kW

Hot

Hot

4 kW
3 kW 4 kW

Cold

12 kW
5 kW 4 kW

Hot

4 Ton AC

4 Ton AC

4 Ton AC

R. AC

4 Ton AC

Fig: Example

Some important checking and then also measurement data analysis:

Air conditioning unit: Measured supply and return temperatures (Preferable difference > 8 C ) Room temperatures measurement Rack temperatures and airflow measurement Aisle & floor tile arrangement Condition of subfloors

Fig: Supply & Return Temperature of Aircon

Fig: Rack temperature checking

Fig: Room Temperature checking points