Stanford Server and Telecom Rooms Design Guide - Final - 2011!01!27

StanfordUniversityServer/TelecomRooms
DesignGuide
1.0
2.0
3.0
4.0
5.0
6.0
7.0
CONTENTS
HowtoUsethisGuide...........................................................................................3
1.1
DocumentOrganization.............................................................................3
SummaryFormandSampleOptionsMatrix.........................................................3
OptionsMatrix.......................................................................................................7
RecommendedCoolingSystemsandRoomLayoutsbySpaceType....................8
TypesofRoomsEnvironmental,Reliability,andAvailabilityRequirements......10
5.1
2008ASHRAEGuidelinesandITReliability...............................................10
5.2
HumidityControl,AirRecirculation,andOutsideAirEconomizers..........12
5.3
TypesofRooms.........................................................................................12
ResearchComputing/HighPerformanceComputing...........................12
Administrativeservers/Lowdensityracks.................................................12
Mixeduse(Researchcomputing/administrative).....................................13
TelecommunicationsRooms.....................................................................13
AirManagementandAirDistribution...................................................................13
6.1
AirManagement........................................................................................13
6.2
AirDistribution...........................................................................................16
CoolingSystems......................................................................................................17
7.1
Houseair....................................................................................................17
7.2
Chilledwaterairhandlingunit(s)withoutsideaireconomization............17
7.3
Chilledwatercomputerroomairhandlingunits(CRAH)..........................18
7.4
ChilledwaterInrackcooling.....................................................................18
7.5
ChilledwaterInrowcooling.....................................................................19
7.6
Passivereardoorcoolingcoil....................................................................19
7.7
Directexpansion(DX)computerroomairconditioningunits...................19
7.8
SplitACsystems.........................................................................................20
7.9
Chilledwaterfancoilunit..........................................................................20
7.10 Exhaustfanwithundercutdoor................................................................20
7.11 Consideranalternatelocationwithchilledwateraccess.........................21
StanfordUniversityServer/TelecomRoomsDesignGuideJanuary,2011
page1

8.0
9.0
10.0
11.0
Controls.................................................................................................................21
8.1
InterfacingwithStanfordsEnergyManagementControlSystem..........21
8.2
AirTemperature.......................................................................................21
8.3
VariableSpeedDrives...............................................................................22
8.4
HumidificationSystemsandControls.......................................................22
MonitoringandSubMetering..............................................................................23
9.1
OnSiteMonitoringandContinuousPerformanceMeasurement............23
9.2
ITLoad.......................................................................................................23
9.3
Electrical....................................................................................................24
9.4
ChilledWater.............................................................................................24
ElectricalEquipment..............................................................................................25
10.1 ElectricalSystems......................................................................................25
10.2 PowerDistributionUnits(PDU).................................................................25
10.3 UninterruptiblePowerSupplies(UPS).......................................................25
10.4 Generators.................................................................................................27
PowerUsageEffectiveness(PUE)..........................................................................27
page2
1.0
Howtousethisguide
ThisguidewasdevelopedtoassistStanfordFacilities,contractors,designengineers,andinformation
technology(IT)endusersintheenergyefficientandcosteffectivedesignandconstructionofserver
andtelecommunicationsspacesontheStanfordcampus.Theguidepertainstobothnewconstruction
andsubstantialretrofits,andprovidesaframeworkfordecisionmakingthroughoutthedesignand
constructionprocess.Guidanceisprovidedonrecommendedcoolingsystems,ITroomlayout,air
managementandairdistribution,controls,monitoring,andelectricaldistribution.Asummaryformis
providedinthenextsectiontoensurethestepsprescribedinthisdocumenthavebeenfollowed,and
guidanceisprovidedtoweighoptionsduringthedesignprocess.
ThisdocumentisintendedtobeusedwithallexistingFacilitiesDesignGuides(FDG)atStanford.Fora
completelistofcurrentFDGs,pleasereferto:http://maps.stanford.edu/fdg_available
1.1
DocumentOverview
ServerandTelecomRoomsDesignGuideSummaryFormwithsampleoptions/decisionmatrix
forevaluatingalternatives
Tableofrecommendedcoolingsystemsandroomlayoutsbyspacetype
Descriptionsofspacetypesandenvironmentalparameters
Guidanceonbestpracticesinserver/telecomroomairdistributionandairmanagement
Descriptionsofrecommendedandalternativecoolingsystems
Guidanceonsystemcontrols,monitoring,andsubmetering
Guidanceonelectricaldistributiondesignandimplementation
Guidanceoncalculatingpowerusageeffectiveness(PUE)
2.0
SummaryFormandSampleOptionsMatrix
ThefollowingthreepagesconsistofasummaryformtoensurethatdesignteamsandStanfordProject
ManagersfollowtherecommendationsoutlinedinthisServerandTelecomRoomsDesignGuide.Itis
theresponsibilityoftheStanfordPM,mechanicalengineerofrecord,electricalengineerofrecord,
and/orarchitectofrecord(ifapplicable)toworktogethertocompletetheforminthespacesprovided.
Anyspecialconsiderationsorparticularprojectconstraintsthatpreventthedesignteamfromfollowing
therecommendationsinthisguideshouldbedescribedinthespacesavailable(orattachedtotheform
asaseparatedocument).Oncecompleted,thesummaryformshouldbesubmittedalongwiththeForm
1processasapartoftheinitialprojectfunding.Toaidinevaluatingalternatives,anoptionsmatrixis
provided.Formoreinformationonhowtousetheoptionsmatrix,refertosection3.0.
page3
StanfordUniversityServerandTelecommunicationsRooms
DesignGuideSummaryForm
page1of3
ThisformshouldbesubmittedalongwithForm1.
ProjectName/Title______________________________________________________________
ProjectLocation_________________________________________________________________
StanfordProjectManager_________________________________________________________
MechanicalEngineerofRecord_______________________________________________________
ElectricalEngineerofRecord_________________________________________________________
ArchitectofRecord__________________________________________________________________
DesignInformation:
Design(peak)ITload_________________________kW
(Serverand/orTelecomLoadsonly)
FloorArea__________________________________ft2
NumberofITRacks___________________________
DesignITLoadDensity________________________W/ft2
DesignITRackDensity_________________________kW/rack
TotalDesignCoolingLoad_____________________tons
(includesITequip,lights,people,etc)
ASHRAEAllowableEnvironmentalConditions(Section5.1):
ASHRAEClassDesignation(circleone):
Class1
Class2
Class3
Class4
TargettemperaturerangeforAirenteringITequipment:
Low:___________oFdrybulbHigh:___________oFdrybulb
Willtheserver/telecomspacerequireactivehumiditycontrol(circleone)?Yes No
Ifyes,fillinhumiditycontrolparametersbelow;ifno,leaveblank.
LowestallowablemoisturecontentofairenteringITequipment_____%RHor____oFdewpoint
HighestallowablemoisturecontentofairenteringITequipment_____%RHor____oFdewpoint
CoolingSystemsSelection(Section4.0):
Doestheprojecthaveviableaccesstochilledwaterfromthecentralplant(circleone)?Yes
No
Ifyes,canthecoolingloadsbeservedbyreturnchilledwater(circleone)?YesNo
Ifno,explain:______________________________________________________________________
Doesthespacehaveviableaccesstooutsideairforeconomization(circleone)?Yes
No
Ifno,explain:______________________________________________________________________
page4
SelectedCoolingSystems:page2of3
A.___________________________OptionsMatrixScore:_____________CircleRanking:123
B.___________________________OptionsMatrixScore:_____________CircleRanking:123
C.___________________________OptionsMatrixScore:_____________CircleRanking:123
Room/equipment/airdistributionlayout(Section6.0):
Checkallthatapply:
Hotaisle/coldaisle
AirSupply:
AirReturn:
Sealed
RaisedFloor
CeilingPlenum
Unsealed
DuctedOverhead
DuctedOverhead
Nohot/coldaisle
Free(unducted,noplenum
Inrackorinrowcoolingsystemlayout
Room/equipment/airdistributionlayoutdeterminednottobeafactor
Other(describe)__________________________________________________________________
ElectricalSystemsSelection(Section10.0):
UPSSystemType:
UPSRedundancy:_________________________
Standby
NumberofUnits:__________________________
LineInteractive
CapacityperUnit:__________________________
DoubleConversion
Rotary/Flywheel
Other(describe)___________________________________________
ExpectedUPSloadfactor:
Atprojectcompletion(dayoneoperation):___________%permodule
Atpeakelectricalloading(fulldesigncapacity):___________%permodule
PowerDistributionUnits(PDU):
PDUscontaininternaltransformers(circleone)?Yes
No
TotalnumberofPDUs:__________________Anticipatedloadfactor:______________________%
Controls,Monitoring,andSubMetering(Sections8.0and9.0):
WillthecoolingsystemsbeconnectedtotheStanfordCampusEMCS? Yes
No
Describeanyrequiredsubmeteringformeasurement/verificationofperformance/PUE:___________
___________________________________________________________________________________
___________________________________________________________________________________
page5
CalculatedPUE(Section11.0): Page3of3
Atprojectcompletion(dayoneoperation):___________________
Atpeakelectricalloading(fulldesigncapacity):___________________
Timeestimatefromdayonetofullcapacity:______________monthsyears (circleone)
Coordination:
HasStanfordS.E.M.Utilitiesevaluatedtheimpactofadditionalloadsontheexistinginfrastructure?
Thermo:YesNo(circleone)
Electrical:YesNo(circleone)
HasanElectricServiceApplicationandInformationSheetbeensubmittedtoStanfordS.E.M.Utilities?
YesNo(circleone)Linktoform:http://lbre.stanford.edu/sem/electric_service
OptionsMatrixTemplate:
Criteria
Relativeenergycostsavings
System1
Criteria
weight
Rating
Rating
Notes
System3
Rating
LifeCycleCostAnalysis(5yr,
4.9%realdiscountrate)
Flexibility(future
reconfiguration)
Reliability(failuremodes,
redundancy)
Firstcost
Otherissues
100
Score
Notes
Notes
System2
page6
3.0
OptionsMatrix
WhenconsultingTable2todeterminerecommendedcoolingsystems,therearegenerallythreechoices
foranygivenloadinglevel/spacetype.Theoptionsarepresentedinorderofmosttoleastenergy
efficient,butoftenthereareotherfactorsbesidesenergyperformancetoconsider.Stanford
recommendsthatdesignersandcontractorsweighthevariousoptionsbaseduponthefollowing
criteria:energycostsavings,firstcost,lifecyclecost,flexibility,reliability,andother(tocaptureany
uniqueconstraintsofaparticularproject).Tothisend,anexampleOptionsMatrixhasbeenfilledout,
andisshowninTable1below(note:eachRatingwillvarybyproject,andCriteriaWeightare
recommendationsforStanford,butcanvary).
System1
System2
System3
Criteria
Criteria
weight
Rating
Relativeenergycostsavings
15
Firstcost
10
LifeCycleCostAnalysis(5yr,
4.9%realdiscountrate)
25
Flexibility(future
reconfiguration)
15
Reliability(failuremodes,
redundancy)
25
Otherissues
10
10
Notes
Rating
5
7
0
100
Score
Notes
Rating
Notes
6
6
0
600
455
595
Table1:Exampleoptionsmatrixforevaluatingalternatives
TousetheOptionsMatrix,coordinatewithallprojectstakeholderstodevelopthecriteriaweighting
factorsforeachofthecriteriaintheleftmostcolumn.FortheexampleinTable1,Stanfords
recommendedweightingcriteriaisprovided,butthesewillultimatelydependontheconsensusreached
byallstakeholdersonwhichofthecriteriaaremostimportanttotheindividualproject.Oncethe
weightingoftheindividualcriteriaisdetermined,thenextstepistorateeachproposedsystem(with
calculationsandbackupdocumentation)onascaleof1to10(1beingtheworstand10beingthebest).
Oncetheratingsarealldetermined,theyaremultipliedbythecorrespondingweightingfactorand
summedtoprovideatotalscore.Thesystemwiththehighesttotalscoreisthenthesystemofchoice.In
theexampleinTable1,System1hasthehighestscorewith600,butSystem3isaveryclosesecond
page7
with595.Inthiscasethematrixhashelpednarrowdownthechoicesfromthreetotwo,andfurther
refinedanalysisshouldhelpnarrowthosetworemainingchoicesdowntotheone,finalchoice.Note:
theOptionsMatrixcanbeextendedtoothersystemsasidefromcoolingsystems,suchasUPSunits,
transformers,lightfixtures,ITequipment,andanythingelseinwhichseveraloptionsaretobe
considered.
4.0
RecommendedCoolingSystemsandRoomLayoutsbySpaceType
Table2belowisintendedtoprovidedesignguidanceforStanfordITfacilities.Tousethetable,firstfind
thespacetypeandloadingthatmostrepresentstheproject.Thebreakdownisbasedupontotalroom
load,withresearch/highperformancebrokenintohighdensityandlowdensity.Highdensitycanbe
takenroughlytomeananyspaceswithadesignloadofapproximately100Wattspersquarefoot(W/sf)
orhigher.
Whenconsultingthetable,firstdeterminetheload/spacetype,andthendetermineifthelocationhas
accesstochilledwater.Approximately60%ofthebuildingsontheStanfordcampushaveaccessto
chilledwaterfromthecentralplant,and(fromanenergyefficiencyperspective)chilledwatercoolingis
muchmoreefficientthanaircooledsystemssuchasDXcomputerroomairconditioning(CRAC)units,or
DXsplitACsystems.Yetanotherefficiencygaincancomefromtheuseofreturnchilledwater,as
opposedtosupplychilledwater.Designingachilledwaterbasedcoolingsystemtorelyonanelevated
chilledwatertemperaturefromthereturnlinebacktothecentralplant(approximately5560oF)can
takeadvantageofavailablechilledwaterthathasalreadydonesomeusefulcooling,enhanceheat
recoverysystemsatthecentralplant,andthehigherdesignsupplyairtemperaturesallowformore
hoursoftheyearofoutsideaireconomization.
Ifthereischilledwaterservice,refertotherecommendedcolumn,andifthereisnochilledwater
available,refertothealternativecolumn.Allrecommendedsystemsemploychilledwater,and
alternativesystemsassumethereisnochilledwateraccess(anditisnotcosteffectivetoprovideit).
Thereareafewcases,namelycomputingloadsgreaterthan500kW,inwhichchilledwatercooling
systemsarehighlyrecommended,andifthereisnochilledwateraccessitisadvisedthatanalternative
locationbeexploredthatcanusechilledwater.Alternatively,thesecomputingresourcesmaybeableto
belocatedatForsytheHall,whichalreadyuseshighlyefficientchilledwaterbasedcoolingsystems.
Withineachcell,youwillfindonetothreecoolingsystems,rankedinorderofenergyperformance.
Eachspace/loadtypeisalsoprovidedwitharecommendedroomlayout.Refertosections6.0and7.0
forfurtherinformationoncoolingsystemsandroomlayout.ThefinalcolumninTable2givesatarget
rangeforpowerusageeffectiveness(PUE).FormoreinformationoncalculatingPUE,refertosection
11.0.
page8
Design IT Load
(Total Room)
Typical IT
Equipment
Recommended Cooling
System and Room
Layout (assumes CHW
access)
System
Layout
Alternative Cooling
System and Room Layout
(assumes no CHW access)
System
Target Power
Usage
Effectiveness
(PUE) Range
Layout
Computing
0 5kW
Admin
1. EF
2. AHUOSA
3. FCU
1. EF
2. HA
3. SAC
1.2 2.0
Computing
5 20kW
Admin
1. EF
2. AHUOSA
3. FCU
1. EF
2. HA
3. SAC
1.2 - 1.8
Computing
20 100kW
Admin
1. AHUOSA
2. CRAH
3. FCU
1. CRAC
2. SAC
3. Alt-CHW
1.2 - 1.6
Computing
100 500kW (Low
Density)
Admin,
Research,
High
Performance
1. AHUOSA
2. CRAH
3. FCU
1. CRAC
2. SAC
3. Alt-CHW
1.1 1.5
Computing
100 500kW
(High Density)
Research,
High
Performance
1. IRAC
2. IROC
3. PCC
1. CRAC
2. SAC
3. Alt-CHW
1.1 1.5
Computing
>500 kW (Low
Density)
Research,
High
Performance
1. AHUOSA
2. CRAH
3. FCU
Alt-CHW
n/a
1.1 1.5
Computing
>500 kW (High
Density)
Research,
High
Performance
1. IRAC
2. IROC
3. PCC
Alt-CHW
n/a
1.1 1.5
Network/
Telcom
0 20 kW
Networking,
Telecom
1. EF
2. AHUOSA
3. FCU
1. HA
2. SAC
3. CRAC
1.2 2.0
Network/
Telcom
> 20 kW
Networking,
Telecom
1. AHUOSA
2. CRAH
3. FCU
1. HA
2. SAC
3. CRAC
1.2 - 1.6
Table2:Recommendedandalternativecoolingsystemsandroomlayoutbaseduponspacetypeandloading.
Keytoabbreviations:
HA=HouseAir
IRAC=chilledwaterinrackcooling
IROC=chilledwaterinrowcooling,
CRAC=computerroomairconditioner
FCU=chilledwaterfancoilunit
1=roomlayoutmaynotbeasignificantfactor
3=hotaisle/coldaislewithaislesealing
AHUOSA=chilledwaterAHUwithoutsideaireconomizer
CRAH=chilledwatercomputerroomairhandler
PCC=passivecoolingcoil
SAC=SplitDXACsystem
EF=exhaustfanwithundercutdoor/louver
2=hotaisle/coldaislewithoutaislesealing
AltCHW=consideranalternatelocationwithchilledwater
access.
page9
5.0
Typesofrooms:Environmental,Reliability,andAvailabilityRequirements
5.1
2008ASHRAEGuidelinesandITReliability
Oneofthefirststepsindesigningthecoolingandairmanagementsystemsinadataspaceistoreferto
thestandardizedoperatingenvironmentsforITequipmentsetforthbytheAmericanSocietyofHeating,
Refrigerating and AirConditioning Engineers (ASHRAE). In 2008, ASHRAE, in collaboration with IT
equipmentmanufacturers,expandedtheirrecommendedenvironmentalenvelopeforinletairentering
IT equipment. The revision of this envelope allows greater flexibility in facility operations, and
contributes to reducing the overall energy consumption. The expanded, recommended and allowable
envelopes for Class 1 and 2 data spaces are shown in Figure 1 and Tabulated in Table 3. More
informationregardingtemperatureandhumiditycontrolcanbefoundinSection8.0.
Figure1:2008ASHRAEenvironmentalenvelopesforITequipmentairintakeconditions
page10
Class1andClass2
Recommended
Range
Class1Allowable
Range
Class2Allowable
Range
LowTemperatureLimit
64.4FDB
59FDB
50FDB
HighTemperature
Limit
80.6FDB
89.6FDB
95FDB
LowMoistureLimit
41.9FDP
20%RH
20%RH
HighMoistureLimit
60%RH&59FDP
80%RH&62.6FDP
80%RH&69.8FDP
Table3:ASHRAERecommendedandAllowableInletAirConditionsforClass1and2DataSpaces
ASHRAEsclassdefinitionsareasfollows:
Class 1: Typically a data space with tightly controlled environmental parameters (dew point,
temperature,andrelativehumidity)andmission/researchcriticaloperations;typesofproductstypically
designedforthisenvironmentareenterpriseserversandstorageproducts.
Class 2: Typically an information technology space or office or lab environment with some control of
environmentalparameters(dewpoint,temperature,andrelativehumidity);typesofproductstypically
designed for these environments are small servers, storage products, personal computers, and
workstations.
Class 3: Typically an office, home, or transportable environment with little control of environmental
parameters(temperatureonly);typesofproductstypicallydesignedforthisenvironmentarepersonal
computers,workstations,laptops,andprinters.
Class 4: Typically a pointofsale or light industrial or factory environment with weather protection,
sufficientwinterheating,andventilation;typesofproductstypicallydesignedforthisenvironmentare
pointofsaleequipment,ruggedizedcontrollers,orcomputersandPDAs.
ThevastmajorityofdataspacesatStanfordaddressedinthisguidewillfallundereitherclass1orclass
2designations,andthusarethefocusofthisguide.
page11
StanfordRecommendations:
Tomaximizeenergyefficiencyperformance,itisstronglyrecommendedthatserver/telecomroom
coolingsystemsbedesignedtomeettheASHRAEallowabletemperaturerange,andthathumidity
notbecontrolledatall(seenextsection).Designingcoolingsystemsforhighersupplyair
temperaturesalsoallowsformanymorehoursinatypicalyearinwhichairsideeconomizationis
viable,andallowsfortheuseofhighertemperaturereturnchilledwaterfromtheStanfordcentral
utilityplant(ifavailable).
5.2
HumidityControl,AirRecirculation,andOutsideAirEconomizers
Humidification and dehumidification are very energyintensive processes. For dehumidification, often
very low chilled water temperatures (42oF to 45oF) are required to lower the air to its dew point
temperature,sacrificingenergyperformanceatthechillerplant.Forhumidification,manytechnologies
employelectricresistanceheaterstoboilwater,anenergyintensiveprocessthatactuallyaddsheattoa
roomthatisincooling.Adiabatichumidifiersemployingultrasonicorevaporativeprocessesareabetter
choice from an energyefficiency perspective. But in the relatively mild climate at Stanford, in which
even the outside air (let alone interior spaces) rarely has a relative humidity outside of the allowable
rangeprescribedbyASHRAE,humiditydoesntneedtobetightlycontrolled.Forsystemsthatrecirculate
themajorityofthecoolingair(i.e.nooutsideaireconomization),thereisverylittlemoistureaddedor
removedduringtherecirculation(exceptinthecaseoflowtemperaturechilledwatercausingunwanted
dehumidification), and the notion that humidity should be controlled at all should be challenged. For
spaces requiring tight humidity control, and for control of outside air economizers during humidity
extremes,centralizingthecontrolandemployingadiabatichumidificationarehighlyrecommendedfor
optimumenergyperformance.
Formoreonhumidityandtemperaturecontrol,refertoSection8.0ofthisguide.
5.3
TypesofRooms
ResearchComputing/HighPerformanceComputing
Generally requiring relatively tight environmental controls, research and high performance computing
facilitiestypicallyfallintothecategoryofaClass1dataspace.Thesetypesofroomshavethehighest
loaddensities,withthehighestdensitiesreaching1,000Wattspersquarefoot,or40KW/rack.These
spaces are generally most efficiently cooled with closecoupled, chilled waterbased cooling. Direct
expansion(DX)coolingsystemsaregenerallynotrecommendedforthesespacesduetotheirrelatively
poor energy performance, and proper air management techniques (see subsequent Air Management
sectionofthisguide)areimportanttoensureefficientdeliveryofcoolingairandextractionofheat.
Administrativeservers/Lowdensityracks
Administrative servers and low density racks generally consist of storage devices, small servers and
workstations. These spaces generally fall into the Class 2 category, with less strict environmental
controlsandreliabilityrequirements.Theoverallload,andloaddensityareusuallylowerthanClass1
page12
spaces. These spaces can be efficiently cooled using more common and established airbased cooling
systems, and for optimum energy efficiency chilled water and outside airbased systems should be
consideredwheneverpossibletomaximizeenergyperformance.
Mixeduse(Researchcomputing/administrative)
Acombinationofbothoftheabovementionedroomtypesinonespaceshouldfollowtheclass
designationofthemostcriticalloads.Inordertoeffectivelycoolthesespacesitisimportantto
implementzonecontainmentmethods(e.g.barriers,stripcurtains)toseparateareasofhighandlow
density,anddirectaproportionalamountofcoolingairtothesespacesbasedupontheloadsbeing
served.Anotheroptionistoimplementmorethanonecoolingsysteminthespace.Forexample,chilled
watercomputerroomairhandlingunits(oranairhandlingunitwithoutsideaireconomization)canbe
usedtoservethelowerdensityportionsofthespace,whileinrack,inrow,andotherclosecoupled
coolingstrategiescanbeemployedinthehigherdensitylocations.
TelecommunicationsRooms
TelecomroomswilltypicallyfallunderClass2,butinsomecasesmayberequiredtobeClass1.Atypical
telecomroomwillcontain15equipmentrackswithupto10kWperrack.Theequipmentmayhave
differentairflowpatternswhichwillneedtobetakenintoaccountwhendesigningthecoolingsystem.
Duetotherelativelysmallsizeofthesespaces(whencomparedtolargeserverrooms),itmaynotbe
necessarytoarrangeequipmentracksinahotaisle/coldaisleconfigurationforthecoolingairflowand
inmostcasesasimpleexhaustfan(thermostaticallycontrolledtopreventcontinuousoperation)can
generallysufficetoremovetheheatfromtheroom.Astelecomroomsbecomelargerandmore
denselypackedwithheatgeneratingequipment,theyshouldbetreatedassmallserverroomswith
properairmanagementandairdistribution(describedinthenextsection).
6.0
AirManagementandAirDistribution
6.1
AirManagement
Airmanagementforserverandlargertelecomroomsentailsallthedesignandconfigurationdetailsthat
gointominimizingoreliminatingthebypassofcoolingairaroundrackintakesandtherecirculationof
heat exhaust back into rack intakes. When designed correctly, an air management system can reduce
operating costs, reduce first cost equipment investment, increase power density (Watts/square foot),
and reduce heat related processing interruptions or failures. A few key design issues include the
configurationofequipmentsairintakeandheatexhaustports,thelocationofsupplyandreturns,the
largescaleairflowpatternsintheroom,andthetemperaturesetpointsoftheairflow.
ImplementCableManagement
For raised floor applications, a server/telecom room should have a cable management strategy to
minimize air flow obstructions caused by cables and wiring. Instituting a cable mining program (i.e. a
programtoremoveabandonedorinoperablecables)aspartofanongoingcablemanagementplanwill
helpoptimizetheairdeliveryperformanceofthecoolingsystems.Thisstrategyshouldtargettheentire
page13
coolingairflowpath.Cablecongestioninraisedfloorplenumscansharplyreduce thetotalairflowas
well as degrade the airflow distribution through the perforated floor tiles. Both effects promote the
development of hot spots. A minimum effective (clear) height of 24 inches or greater should be
providedforraisedfloorinstallations,creatingamoreuniformpressuredistribution.
AisleSeparationandContainment
Inahotaisle/coldaisleconfiguration,allequipmentisinstalledintotherackstoachieveafronttoback
airflowpatternthatdrawsconditionedairinfromcoldaisles(airintakeinfrontoftheracks)andrejects
heat out through the hot aisles (air heat exhaust behind the racks). Equipment with nonstandard
exhaust directions must be addressed in some way (shrouds, ducts, etc.) to achieve a fronttoback
airflow.Rowsofracksareplacedbacktoback,vacantequipmentslotsareblockedofftocreatebarriers
reducingrecirculation,andcableopeningsinraisedfloorsandceilingsaresealedastightlyaspossible.
Onerecommendeddesignconfiguration,showninFigure2,suppliescoolairtotheracksviaanunder
floorplenum(alternativelycouldbeductedoverhead);theairthenpassesthroughtherackandentersa
semisealedareaforreturntoanoverheadplenum(orreturnduct).Abafflepanelorbarrierabovethe
topoftheracksandattheendsofthehotaislesmitigatesshortcircuiting.Theseimprovementshave
beenshowntoreducefanenergyrequirementsby20%to25%(ifequippedforvariableairflow),and
couldresultina20%energysavingsatthecoolingsystem.Notethatwhilethefigurerepresentsaraised
floorsupplyplenumarrangement,thesameconfigurationcanbeaccomplished(oftenmoreefficiently)
usingductedoverheadsupplyandreturnair(i.e.noraisedfloor).
Any proposed design features need approval by building/fire officials before implementation.
Sometimesthefullsealingofaislescantriggerfirecodeissues.Variancesareoftengrantedwhenthe
sealingofaislescanbeshowntonothaveanegativeeffectonfiresuppressionsystems.Incaseswhere
thesevariancesarenotgranted,onesolutionistopartiallysealthetopsandsidesoftherows,leaving
thenecessaryfreespaceforactivationoffiresuppressionsystems.
Figure2:Sealedhotaisle/coldaisleconfiguration
page14
OptimizeSupplyandReturnAirConfiguration
Hot aisle/cold aisle configurations can be served by overhead or underfloor air distribution systems.
Overheaddeliverytendstoreducetemperaturestratificationincoldaislesascomparedtounderfloor
airdelivery.Whenanoverheadsystemisused,supplyoutletsthatdumptheairdirectlydownshould
be used and located directly in front of racks, above the cold aisle. The temperature monitoring to
controlthecoolingunitsshouldbelocatedinareasinfrontofthecomputerequipment(ITequipment
intakes).Useofvariableairvolume(VAV)allowsequipmenttobesizedforexcesscapacity,yetprovide
optimizedoperationatpartloadconditionswithturndownofvariablespeedfans.
In an underfloor air supply system, the underfloor plenum often serves both as a duct and a wiring
chase.Coordinationthroughoutdesignandintoconstructionandoperationthroughoutthelifeofthe
facilityisnecessarysince pathsforairflowcan be blockedby electricalordatatraysand conduit.The
location of supply tiles needs to be carefully considered to prevent short circuiting of supply air and
checkedperiodicallyifusersarelikelytoreconfigurethem.Removingoraddingtilestofixhotspotscan
causeproblemsthroughoutthesystem.Anotherimportantconcerntobeawareofishighairvelocityin
theunderfloorplenum.Thiscancreatelocalizednegativepressureandinduceroomairbackintothe
underfloor plenum. Deeper plenums (>24 inches) and careful layout of CRAC/CRAH units allow for a
moreuniformunderfloorairstaticpressure.
Onerelativelyeasy,inexpensive,andeffectiveretrofitfordownflow(returnairentersthroughthetop
oftheunitandsupplyairisviaunderfloorplenumorlowprofilesupplyoutlet)CRACorCRAHunitsisto
utilizetheaboveceilingplenum(seereturnairconfigurationinFigure2)andconnecttheplenumtothe
returnairopeningsoftheCRAC/CRAHunits.Thishelpstomitigateshortcircuitingofthesupplyairinto
thereturnopenings.AnexampleisshowninFigure3.
Figure3:DuctingfromceilingplenumairreturntoopeningofCRAHunit.
page15
RaisingTemperatureSetPoints
A higher supply air temperature and temperature difference between the return and supply air
increasesthemaximumloaddensitypossibleinthespace,and/orreducesthesizeoftheairsidecooling
equipmentrequired.Theresultinglowerrequiredsupplyairflowprovidestheopportunityforfanenergy
savingsandcaneasetheimplementationofanairsideeconomizer.
Inordertosaveenergyusinganairsideeconomizer,thetemperatureoutsideonlyhastobecoolerthan
the return air that is exhausted from the room. As the return air temperature is increased, the
temperatureatwhichanairsideeconomizerwillsaveenergyisincreased.Designingforahigherreturn
air temperature increases the number of hours that outside air can be used to provide the necessary
coolingandsaveenergy.
AirflowEfficiency
Thismetriccharacterizesoverallairflowefficiencyintermsofthetotalfanpowerrequiredperunitof
airflow. This metric provides an overall measure of how efficiently air is moved through the
server/telecomroom,fromthesupplytothereturn,andtakesintoaccountlowpressuredropdesignas
wellasfansystemefficiency.ThecommonmetricforairflowefficiencyistheWattsoftotalfanpower
per cubic feet per minute of airflow (W/cfm). Table 4 gives some benchmarking values for airflow
efficiency.
Standard
Good
Better
1.25 W/cfm 0.75 W/cfm 0.5 W/cfm
Table4:Airflowefficiencybenchmarking
6.2
AirDistributionMethods
RaisedFloor
Themainobjectiveistocollectwasteheatascloseaspossibletothesourceandthenisolateitfromthe
spaceinthereturnairstream.Floorsupplytilesarereplacedbysolidtilesinthelowloadareato
increasetheairsuppliedtothefarthest,andlargest,rackloads.
Thisguidedoesnotrecommendtheinstallationofaraisedfloorinabrandnewdataspace.Ithasbeen
shownthatanoverheadductingsystemcanprovidejustasefficient,ifnotmoreefficientairdistribution
withouttheaddedexpenseofaraisedfloorinstallation.Discussionsofraisedfloorinstallationsare
includedforcasesinwhicharetrofitisbeingundertakeninaspacewithapreexistingraisedfloor.
Overhead
Inanoverheaddistributionsystem,thesupplyoutletsaredesignedtodumptheairdirectlydowninto
thecoldaisles.ThisiscountertotraditionalofficeHVACdesign,inwhichsupplyairdiffusersare
designedtoencourageairmixinginthespace.Inahotaisle/coldaisleconfiguration,thesupplyoutlets
arelocatedabovethecoldaisle,infrontoftheracks.Returngrillesarelocatedinthehotaisles.
page16
Overheaddeliverytendstoreducetemperaturestratificationincoldaislesascomparedtounderfloor
airdelivery.
InRack/InRowSystems
Theairdistributionmethodsemployedbyinrack/inrowcoolingsystemsvarywidelybymanufacturer
andmodel.Consultwiththespecificmanufacturersrepresentativeforrecommendationsonthe
optimalairdistribution/roomlayoutforaparticularinrack/inrowsystem.
7.0
CoolingSystems
ThefollowingdescribestherecommendedandalternativecoolingsystemsoutlinedinTable2,andthe
abbreviationsinparenthesescorrespondtotheabbreviationsusedinthetable.
7.1
Houseair(HA)
Whensmallserverortelecomroomsareaddedtoexistingbuildingspaces(andsometimeseveninnew
construction),arelativelyeasymethodofprovidingthenecessarycoolingistotapintoanexisting,
oftencentralizedHVACsystem.Thissolutiongenerallyconsistsofductingtothemainsupplyandreturn
ductsandplacingsupplyoutletsandreturngrillesintheserver/telecomspace.
Whilethiscoolingsolutionmayseemlikeabeneficialonefromafirstcostandconstructability
perspective,whatoftenhappensisthatanentirecentralizedbuildingHVACsystemendsupbeing
requiredtoruncontinuouslyjusttoserveafewsmallspaceswithcontinuouscoolingloads.Infacilities
thatrequirecontinuousoperationofthecentralHVACsystems,thisisgenerallynotaproblem.Butif
themainbuildingHVACsystemisabletorunonaregularoccupancyschedule(e.g.Mondaythrough
Friday,7am7pm),requiringthesystemtoruncontinuouslyjusttoserveafewsmallspacesturnsout
tobeanenergywaste.
Onehybridoptionentailstheuseofthishouseairduringregularlyoccupiedhours,whichthen
switchesovertoadedicatedcoolingsystemduringunoccupiedhours.Theconceptisthatwhilethe
centralsystemisrunning,itismoreefficienttoservethecontinuouscoolingloadswiththecentral
system,butwhenthecentralsystemcanbeturnedoffduringunoccupiedtimes,thededicated
server/telecomroomcoolingsystemiscommandedtorun.Thisdedicatedsystemwillgenerallybeless
efficientthanthecentralsystem,butmuchmoreefficientthanrunningthecentralsystemonlytoserve
afewsmallspaces.
7.2
Chilledwaterairhandlingunit(s)withoutsideaireconomization(AHUOSA)
InthetypicalStanfordclimate,therearemanyhoursinagivenyearwhentheoutsidetemperatureis
conducivetofreecoolingofserverandtelecommunicationscoolingloads.Whiletherearemany
customdesignedmethodstoductoutsideairintoanexistingCRAC/CRAHorevenadirectexpansion
(DX)orchilledwaterfancoil,theeasiestwaytoachievethisinaneworsubstantiallyredesignedserver
ortelecomroomistoinstallairhandlingunits(AHU)withoutsideaireconomizers.Theeconomizerwill
page17
generallycomestandardwiththeunit,preventingtheneedforacustomdesign,althoughsome
modificationstocontrolalgorithmsintendedfornormalofficebuildingsmaybenecessary.
Theenergyefficientchoiceforthecoolingcoil(s)intheAHUischilledwater.Incaseswherechilled
waterisnotavailable,alessefficientsystemcanemployanaircooledorevaporativelycooledpackaged
DXunit(stillwithoutsideaireconomizer).EvenwiththelessefficientDXunit,havingaccesstooutside
aireconomizationduringmanyhoursoftheyear(effectivelynotusingtheDXcooling)stillhasmany
energybenefitsoversystemswithnooutsideairaccess.
7.3
Chilledwatercomputerroomairhandlingunits(CRAH)
Oftenaccesstooutsideair(foreconomizingorfreecooling)isnotreadilyavailable,oritsconstruction
costsareshowntobeprohibitivelyexpensive.Inthesecases,onlytheminimumrequiredventilationair
(foroccupants,ifapplicable)isprovided,andtheserverortelecomroomcoolingisaccomplishedby
recirculationofairthroughacoolingcoilintheserver/telecomspace.Thesearecommonlyreferredto
ascomputerroomairhandling(CRAH)unitsiftheyhavechilledwatercoils,orcomputerroomair
conditioning(CRAC)unitsiftheyemployDXcooling.Themoreefficientchoicefromanenergyefficiency
perspectiveistousechilledwaterCRAHunits.
7.4
ChilledwaterInrackcooling(IRAC)
Inrackcoolingreferstoacoolingsystemthatislocatedwithinanindividualserverrackthatonly
providescoolingtotherackinwhichitiscontained,withthepossibleadditionofservingoneormore
adjacentracks.Oftenchilledwater(sometimesrefrigerant)isprovidedtocoolingcoilsplaceddirectlyin
therackandairiscirculatedbyintegralfanswithinthatindividualserverrack(andpossiblytooneor
moreadjacentracks).Therearemanydifferenttypesandconfigurationsofsuchsystems,andtheyare
generallyagoodapplicationincaseswhererackloaddensitiesareveryhigh.Figure4showsanexample
ofaninrackcoolingunit,buttherearemanyothertypesandconfigurationsavailable.
Figure4:Exampleofaninrackcoolingunit
(Photocredit:AmericanPowerConversionCorp.)
page18
7.5
ChilledwaterInrowcooling(IROC)
Inrowcoolingreferstoacoolingsystemthatislocatedtoserveanindividualrowofserverracks,and
thatonlyprovidescoolingtotherowinwhichitiscontained.Oftenchilledwater(orsometimes
refrigerant)isprovidedtocoolingcoilsplaceddirectlyin/ontherowofserversandairiscirculatedby
integralfans.Thecoolingcanbedonedirectlyintherackitself,orthroughacoolingcoilthatisclose
coupledtotheservers.Therearemanydifferenttypesandconfigurationsofsuchsystems,andtheyare
generallyagoodapplicationincaseswhererackloaddensitiesareveryhigh.Figure5belowillustrates
fourdifferentconfigurationsofinrowcooling.Fromlefttoright,thefirstshowsacoillocatedabovea
hotaisle,withthearrowsshowingcoolingandrecirculationairpaths.Thesecondandthirdillustrations
showacoolingcoilontherearoftheracksinseveralconfigurations,andthelastshowsdirectliquid
coolingtotheserversthemselves.Thebestoptionforaparticularfacilitywilldependhighlyoncooling
load,facilitylayout,andfirstcostconsiderations.
Figure5:Variousconfigurationsofinrowcoolingsystems
7.6
Passivereardoorcoolingcoil(PCC)
Ahighlyefficientoptionforcoolingofserverracksemploysacoolingcoilontherear(exhaust)section
oftheserverracks.Chilledwater(orrefrigerant)isusedinthecoilstoprovidetheneededcooling.
Roomairisinducedintotheserversbyinternalfansintheserversthemselvestherearenoexternal
fans.Theserverfanstheninduceairthroughtheserverstodothenecessarycooling,andwhentheheat
isrejectedfromtherearoftheserver,theairpassesthroughthecoolingcoilbeforebeingintroduced
backintotheroom.Thisairiscontinuouslyrecirculatedwithinthedataspace,andheatisrejectedvia
thecoolingmedium(chilledwaterorrefrigerant).
7.7
Directexpansion(DX)computerroomairconditioningunits(CRAC)
SimilartothediscussionofCRAHunitsabove(Section7.3),thedesignerisoftenfacedwithasituationin
whicha)accesstooutsideairforeconomizationisnotfeasible,andb)thereisnoaccesstochilled
water.InthesecasesCRACunitsareoftenused.Theserecirculatingunitsarelocatedwithinthedata
space,andaremoteoutdoorcondensingunitisemployedtodotheheatrejection.Thecondensingunit
canalsobewatercooled,orevaporativelycooled,slightlyincreasingefficiency,butingeneralDXCRAC
unitsshouldbeavoidedduetotheirrelativelylowenergyperformance.
page19
7.8
SplitACsystems(SAC)
SplitACsystemsrefertoaclassofDXcoolingunitswheretheevaporatorislocatedwithinthe
server/telecomspace,andaremotecondensingunitislocatedoutdoorstorejecttheheat.These
systemsaregenerallyusedtoserverelativelylowcoolingloads(110tons),butdonothavethe
coolingefficiencybenefitsofchilledwaterunits,orlarger,centralizedDXcoolingunits.Splitsystems
alsogenerallydonotcomeequippedwithanoutsideaireconomizer,althoughcustomsolutionscanbe
built.
EfficienciesofsplitACsystemscanvarywidely,andareusuallyexpressedasanenergyefficiencyratio
(EER).EERisdefinedastheratedcoolingoutputinthousandsofBritishthermalunitsperhour(kBtuh)
dividedbytheratedelectricalinputinkilowatts(kW).Splitsystemsshouldgenerallybespecifiedto
haveanEERof14orhigher(dependinguponsystemsize).
Ifthereareseveralsmallserver/telecomroomslocatedwithinareasonabledistancefromoneanother,
itispossibletohavealloftheevaporatorsconnectedtoacommoncondensingunit.Usingvariable
refrigerantflow(VRF,oralternatively,variablerefrigerantvolume,orVRV)technology,multipleunits
canbeservedatvaryingindividualcoolingloads,andsomesystemsevenallowtransferofheatfrom
onespacetoanother.Thisisapotentialformofheatrecoveryinwhichheatcanberemovedfroma
server/telecomroom,andbeusedtoheatforinstance,anearbyofficespace.Figure6showsan
exampleofasplitACsystem.
Figure6:ExampleofasplitDXACsystem(indoorandoutdoorunitsconnectedbyrefrigerantlines)
(Photocredit:MitsubishiElectricCorp.)
7.9
Chilledwaterfancoilunit(FCU)
Forsmallercoolingloadsandheatdensities,forinstanceatelecomroomorsmallserverroom,chilled
waterfancoilunitscanbeaviablecoolingsolution.Theseunitsmayormaynotbeductedtothe
outdoorstoemployairsideeconomizing,andwithintheserver/telecomspacetheairtoandfromthe
unitalsomayormaynotbeducted.Anadvantagetoinstallingchilledwaterfancoilsisthattheyare
moreenergyefficientthanDXsplitsystemsorDXfancoils.
7.10 Exhaustfanwithundercutdoor(EF)
InamildclimatesuchasthatexperiencedatStanford,oftenthemostefficient,andleastexpensive
coolingoptionforsmalltelecomroomsisasimpleexhaustfantoremovetheheatandrejectittothe
outdoors.Toprovidesufficientmakeupair,oftenaportionofthebottomofthedooriscut(toallowfor
page20
moreopenareathroughwhichmakeupaircanentertheroom),oralternatively,afixedlouverinthe
door(showninFigure7)alsoservesthesamepurpose.Implementthermostaticcontrolonthefanto
preventitfromoperatingcontinuously.
Figure7:Exampleoflouverindoortoallowformakeupair
(Photocredit:PolymerConcrete,Inc.)
7.11 Consideranalternatelocationwithchilledwateraccess(AltCHW)
Approximately60%ofthefacilitiesontheStanfordcampushaveaccesstochilledwaterfromthecentral
utilityplant.ChilledwaterbasedcoolingsystemsarealmostalwaysmoreefficientthantheirDX
counterparts.Ifaserverortelecomfacilityisproposedtobesitedoncampusthatisanticipatedtohave
eitherahighcoolingloaddensity(i.e.>100W/sf),orahightotalload(>100kW),considerlocatingthe
facilityinanareathathasaccesstochilledwater.Servinghighcoolingloaddensitiesorhighabsolute
coolingloadswithDXequipmentisgenerallynotanenergyefficientchoiceandshouldbeavoidedif
possible.
8.0
Controls
8.1
InterfacingwithStanfordsEnergyManagementControlSystem
StanfordsEnergyManagementControlSystem(EMCS)shouldbeusedwhereapplicableaspartofthe
control system for any data room applications. The existing controls are an Emerson DeltaV based
system, and are only wired to approximately 100 of the approximately 600 buildings on campus. The
EMCSislargelycapableofmonitoringonly,withactualsystemcontrolbeingaccomplishedbytheEMCS
in a few cases. Designers are encouraged to incorporate new systems into the existing EMCS when
appropriate,andtospecifyequipmentthatiscompatiblewiththeEMCStotheextentpossible.
To determine the extent to which a particular facility, system, or project can be integrated into the
StanfordEMCS,pleasecontacttheStanfordEMCSGroupasearlyaspossibleinthedesignprocess.
8.2
AirTemperature
Usingwarmertemperaturesetpointsinserver/telecomroomsisstronglyencouragedontheStanford
campus due to the corresponding energy efficiency benefits. It is important to consider that in a
properlydesigned server room with best practices in air management, room temperature is a
page21
meaninglessconcept.Coldaisletemperaturescanbeontheorderof65oF85oF,andcorrespondinghot
aisle temperatures can be on the order of 90oF 110oF. Given these parameters, a typical comfort
coolingtemperaturesetpointof72oF78oFisnotinanywayappropriateforaserver/telecomroom.
Thermostats and other sensors that control cooling systems should be located at the inlets to the IT
equipment (cold aisles), as this is the only region of the room in which temperature needs to be
controlled.
Variablespeedfansintheserversaresometimescontrolledtotheinternalservertemperature.Internal
fans may operate at higher speeds, consuming more power, if the server inlet air conditions are
operated above the allowable range. An increase in inlet air temperature results in higher server fan
powerbyapplyingthefanaffinitylawwherefanpowerincreaseswiththecubeoffanspeed.Thus,the
effectofincreasingserverinletairtemperatureonserverfanpowershouldbecarefullyweighedagainst
thepotentialcoolingsystemenergysavings.
8.3
VariableSpeedDrives
Toallowforvaryingairflowasthecoolingloadfluctuates,andtominimizetheairflowforagivencooling
load, variable speed drives should be used on cooling air supply fans (not to be confused with server
fans described in the last section). Variable air volume systems provide consistent cooling when and
whereitisneededandareabletorespondtochangesinload.Thefanspeedshouldbecontrolledto
maintainanunderfloorpressuresetpointinaraisedfloorapplication,orasupplyductstaticpressure
setpointforductedsystems.Supplyairandsupplychilledwatertemperaturesshouldbesetashighas
possiblewhilemaintainingthenecessarycoolingcapacity.
8.4
HumidificationSystemsandControls
Withtheexceptionofspecificneedssuchastapestoragelibraries,activehumidificationcontrolsarenot
implemented as a matter of course in Stanford campus server or telecom rooms. For spaces which
recirculatethemajorityofthecoolingair,thereisnoappreciablesourceofmoisturethatwouldrequire
dehumidification.Barringhighinfiltration,mostcoolingsystemswillautomaticallycontrolthemaximum
humiditytoabout60%to70%RHduetothefundamentalnatureoftheiroperation.Supplyairat60F
saturatedwithasmuchmoistureasitcanpossiblycarryhasarelativehumidityofonly66%RHat72F.
Humidity sensors are possibly the least stable sensor technologies in common HVAC use. Without
regular calibration, attempts to control to a tight humidity band (1% 2%) are not meaningful. The
problemwithexpendinglargeamountsofenergytomaintaina50%RH+/1%whenthehumiditysensor
isoffbyatleast5%isclear.Evenwithanappropriatecontrolbandsuchas2080%RH,humiditysensors
shouldberegularlycalibrated.Humiditycontrolconsumesalargeamountofenergyandrequirescostly
equipment to achieve; if the first cost was justified, then calibrating the humidity sensors should be
consideredasmuchanoperatingcostastheelectricbillinordertoensurethefirstcostinvestmentis
actuallyprovidingthedesiredspacecondition.
page22
9.0
MonitoringandSubMetering
9.1
OnSiteMonitoringandContinuousPerformanceMeasurement
Sufficientmeteringneedstobeinplaceforongoingenergyusagemanagementtobeeffective.Energy
efficiencybenchmarkinggoalsneedtobeestablishedtodeterminewhichmeasuredvaluesneedtobe
obtainedformeasuringefficiency.
Theaccuracyofthemonitoringequipmentshouldbespecified,includingcalibrationstatus,tosupport
thelevelofdesiredaccuracyexpectedfromthemonitoring.Themeasurementrangeshouldbecarefully
consideredwhendeterminingtheminimumsensoraccuracy.Electromagneticflowmetersandstrap
on ultrasonic flow meters are among the most accurate water flow meters available. Three phase
powermetersshouldbeselectedtomeasuretruerootmeansquare(RMS)power.
Ideally,theuseofStanfordsEnergyManagementControlSystem(EMCS)providesallofthesensorsand
calculations required to determine realtime efficiency measurements. All measured values should be
continuouslytrendedanddataarchivedforaminimumofoneyeartoobtainannualenergytotals.Ifan
openprotocolcontrolsystemisinstalled,moresensorsmaybeaddedafterinitialinstallation.IftheIT
equipment includes onboard temperature sensors, it may be possible to integrate them with the
campus EMCS. Measurement should include temperature and humidity sensors at the air inlet of IT
equipmentandatheightsprescribedbyASHRAEsThermalGuidelinesforDataProcessingEnvironments,
2008.
9.2
ITLoad
To get average rack power, count the number of racks in the server/telecom room, and assign a
percentagefullforeachrack.Thepercentagefullmayappeartobesomewhatarbitrary,butitcanbe
veryusefulwhenusedtoestimatefutureloads.Ifdesired,acompleteinventoryoftheITequipment
couldbeobtained,orapartialinventoryoftypicalequipment.Thisinventorycanbeusefulintracking
loadgrowthtrendsandplanningforfutureexpansions.
Thereareseveraltypesofracksthatmaybeofinterest.Selectingatypicalrackallowsforprojectionsof
total rack load and load density, while selecting a peak load rack offers guidance as to the airflow
distributionrequiredforcooling.Ataminimum,measurearackwithatypicalmixtureofequipmentto
getasenseofthevariabilityofthepowerrequirement.Therackneedstohaveitsownelectricalcircuit
andpreferablyjustoneelectricalcircuitfeedingit.Ifdifferenttypesofserversareused,selectarack
withthemostcommontypeofserver.
StanfordRecommendations:
Unlesstherearebuiltinmetersontherackfeedcircuit,installaportablekWloggeratthefeedto
theracks.Ifusingabuiltinmeter,startatrendloginthemonitoringsoftware.Asignificantlength
oftrendtime(usuallyaweek)isrequiredtocapturedataifthereareanyprocessingtaskload
impactsontheoverallpowerusage.
page23
9.3
Electrical
Todetermineelectricalpowerlossesinthedistributionsystem,measuretheinputandoutputpowerat
thesametime.Somedistributionsystemsareequippedwithinternalmeterstoallowthis,thoughthey
should be checked with portable meters to assess their accuracy. Note that kVA is not a reliable
substituteforkW.Designdatacanbesubstitutedifthereisnoeasyaccessforpowermeasurements.
SpecificallyforUPS,determineandrecordtheinputpowerandtheratedmaximumforthesystemand
calculatethepercentloadfactorbydividingtheinputloadbytheratedmaximum.Theloadfactorhasa
significantimpactonthemeasuredUPSefficiency.Lossesinstandbygeneratorsincludejacketheaters
and battery chargers. Measure and log these electrical power loads and the ambient temperature at
thegenerator(s).Alsonotethejacketwatertemperaturesetpointandmeasurethetemperature(one
readingisadequate).
Inordertoreducethenumberofpointsrequiredtodetermine thetotalITequipmentelectricalload,
usethemostupstreamfeedersavailableandfeasibletoaccess,servingonlyITequipment.Typically,a
single meter can be used to capture the power usage of many racks or even a whole (small)
server/telecomroom.
9.4
ChilledWater
SometimesitiseasiesttomonitortheentiremotorcontrolfeedingthechillerplantasonekWpoint,
andanyotherloadsonitcanbemonitoredforsubtraction.PumpswithoutVFDscanbespotchecked
ortrendedforashorttimeandassumedtobeconstantpowerdevices.Thechillerandcoolingtower
systemstendtohavehighvariationsinloadovertimeandshouldbetrended.Variablespeedpumps
shouldalsobetrended.
Inordertooptimizethechillerplant,eachcomponentshouldbeloggedseparately,aswellaschilled
waterandcondenserwatersupplyandreturntemperaturesandchilledwaterflowtoallowforthe
determinationoftonsofcoolingsupplied.Anaccuratetonnagemeasurementrequireshighlyaccurate
temperatureandflowsensors.Thedifferencebetweenthesupplyandreturntemperaturesisoften5
10degreesorless.Useofnonmatched,typicaltemperaturesensorsthatareonlyaccurateto1oF
couldintroduceanuncertaintyofalmost50%intothetemperaturedeltameasurement,renderingit
useless.Waterflowcanbedeterminedbyaclamponultrasonicorbyapermanentlyinstalledmeter,
whichwouldallowforcontinuousmonitoring.Careshouldbetakenwhenselectingaflowmeterforan
installationwithaminimalchanceoffoulingandcalibrationdrift.
ForchilledwaterfromtheStanfordcentralutilityplant,theremayalreadybesufficientsubmeteringof
temperaturesandflowratesforagivenbuildingorroom.Besuretoinvestigateanyalreadyavailable
chilledwatermonitoringsystemsanddatabeforespecifyingsubmeteringforchilledwaterusefora
particularproject.ThisdatamayalsobehelpfulindeterminingthefacilityPUE.
page24
10.0 ElectricalEquipment
10.1 ElectricalSystems
Similartocoolingsystems,itisimportanttoalwaysconsiderinitialandfutureloads,inparticularpart
andlowloadconditions,whendesigningandselectingequipmentforaserver/telecomroomelectrical
system.
Inatypicalserver/telecomroomwithahighlyefficientcoolingsystem,ITequipmentloadscanaccount
foroverhalfoftheentirefacilitysenergyuse.UseofefficientITequipmentwillsignificantlyreduce
theseloads,whichconsequentlywilldownsizetheequipmentneededtocoolthem.Purchasingservers
equippedwithenergyefficientprocessors,fans,andpowersupplies,highefficiencynetwork
equipment,consolidatingstoragedevices,consolidatingpowersupplies,andimplementing
virtualizationarethemostadvantageouswaystoreduceITequipmentloads.
10.2 PowerDistributionUnits(PDU)
Specialty PDUs that convert higher voltage (208V AC or 480V AC) into lower voltage (120V AC) via a
builtinstepdowntransformerforlowvoltageequipmentarecommonlyused.Transformerslosepower
intheformofheatwhenvoltageisbeingconverted.Theparametersofthetransformerinthistypeof
PDUcanbespecifiedsuchthattheenergyefficiencyisoptimized.Adrytypetransformerwitha176F
temperature rise uses 13% to 21% less energy than a 302F rise unit. The higher efficiency 176F
temperatureriseunithasafirstcostpremium;however,thecostisusuallyrecoveredintheenergycost
savings. In addition, transformers operate most efficiently when they are loaded in the 20% to 50%
range.SelectingaPDUwithatransformerloadedinthatrangewillreducethelossofpowerthroughthe
transformer. Energy can also be saved by reducing the number of installed PDUs with builtin
transformers.
10.3 UninterruptiblePowerSupplies(UPS)
A portion of all the power supplied to the UPS to operate the server/telecom equipment is lost to
inefficienciesinthesystem.Thefirststeptominimizetheselossesistoevaluatewhichequipment,ifnot
the entire server/telecom room, requires a UPS system. Increasing the UPS system efficiency offers
direct,24houradayenergysavings,withintheUPSitselfandindirectlythroughlowerheatloadsand
even reduced building transformer losses. Among double conversion systems, UPS efficiency ranges
from 86% to 95%. When a full IT equipment load is served through a UPS system, even a small
improvementintheefficiencyofthesystemcanyieldalargeannualcostsavings.Forexample,a15,000
square foot server room with IT equipment operating at 100 W/sf requires 13,140 MWh of energy
annuallyfortheITequipment.IftheUPSsystemsupplyingthatpowerhasitsefficiencyimprovedfrom
90%to95%,theannualenergybillwillbereducedby768,421kWhorabout$90,000at$0.12/kWh,plus
significant additional cooling system energy savings from the reduced cooling load. For batterybased
UPS systems, use a design approach that keeps the UPS load factor as high as possible. This usually
requiresusingmultiplesmallerunits.
page25
OnestrategyforimprovingUPSenergyefficiencyistoreducetheheatloadsduetopowerconversion.
Power conversion from AC to DC and back to AC occurs with battery UPS systems, resulting in large
energy losses, which are then compounded by the energy for cooling needed to remove the heat
generatedbytheloss.TheefficiencyofthepowerconversiondropsoffsteeplywhentheUPSsystems
arelightlyloaded.
ConsideraUPSsystemsizedfortwoUPSunitswithn+1redundancy,withbothunitsoperatingat30%
loadfactor.Ifthesameloadisservedbythreesmallerunits(alsosizedforn+1redundancy),thenthese
units will operate at 40% load factor. This 10% increase in load factor can result in a 1.2% efficiency
increase(seeFigure8).Fora100kWload,thisefficiencyincreasecanresultinsavingsofapproximately
13,000kWhannually.
Figure8:TypicaldoubleconversionUPSefficiencycurvefor100kVAcapacityandgreater
Evaluate the need for power conditioning. Line interactive systems often provide enough power
conditioning for servers at a higher efficiency than typical double conversion UPS systems. Some
traditionaldoubleconversionUPSsystems(whichofferthehighestdegreeofpowerconditioning)have
theabilitytooperateinthemoreefficientlineconditioningmode,usuallyadvertisedaseconomyor
ecomode.
In some cases a more efficient alternative to traditional, batterybased UPS systems is the use of a
rotary(flywheel)UPS.ThesesystemsaregenerallyimplementedintandemwithabackupDieselengine
generator.Duringnormaloperation,theflywheelspinscontinuously.Intheeventofautilitypowerloss,
the rotational inertia of the highmass rotary element will provide 10 60 seconds of ridethrough
powerasthebackupgeneratorisstartedandrun.Thetransferfromutilitypowertotheflywheel,and
finally to the generator is such that there is no power interruption to critical equipment. Multiple
flywheelsoperatedinparallelcanincreasereserveruntimeandbackuppowercapacity.Thesesystems
page26
have an appreciably longer lifecycle than batterybased UPS systems (on the order of 20 30 years),
butrequireregulardowntimeformaintenancesuchasbearingreplacements.Newersystemsemploying
magneticbearingsrequirefewerreplacementsandresultinlowerstandbylosses.RotaryUPSsystems
alsoeliminatebatteryreplacementanddisposalconcerns,anddontrequiretighttemperaturecontrol
toensurethelongevityofthebatterybanks.
10.4 Generators
Intheeventofapowerfailure,manyserverroomshaveenginedrivengeneratorstobackupthe
system.Suchgeneratorshaverelativelysmallbutsignificantandeasilyaddressedstandbylosses,the
biggestbeingtheengineheater.Theengineheatertypicallyusesmoreenergythanthegeneratorwill
everproduceoverthelifeofthefacility.Reducingheatertemperaturesetpointsfromthetypical120F
to70Fhasshownsavingsof79%withnoreductioninreliability;delayingthetransferswitchtransition
tothegeneratorgreatlyreducesenginewearduringthewarmupperiod.Addingthermostaticcontrol
oftheheater(asopposedtocontinuousonoperation)canalsoprovideforsignificantenergysavings.
FormoreinformationontheuseofDieselenginegeneratorsatStanford,pleaserefertoStanford
UniversityFacilitiesDesignGuide(FDG)#16230.
11.0 PowerUsageEffectiveness(PUE)
Onecommondatacenter/server/telecomroomefficiencymetricisthePowerUsageEffectiveness
(PUE).ThismetricwasdevelopedbytheGreenGrid,aglobalconsortiumofITcompaniesand
professionalsseekingtoimproveenergyefficiencyofserver/telecom/ITfacilities.PUEisausefulmetric
forcomparingsimilarfacilitiesinsimilarclimates,ortotracktheprogressofagivenfacilityovertime.
ButwhilePUEwillshowfacilityoperatorshowefficientlytheyareservingagivenserver/telecomload,it
doesnotevaluatewhetherornotthatserver/telecomloadcouldbereduced(i.e.throughvirtualization,
serverpowermanagement,storageconsolidation,etc.).PUEisdefinedas:
PUE= TotalFacilityPower
PowerforITequipmentonly
Totalfacilitypowerincludesallelectricalusesintheserver/telecomroom,includingtheactual
server/telecomequipment,lights,fans,pumps,coolingsystems(includinganyoutdoorcondensing
units),humidifiers,andlossesintheelectricaldistributionsystem(UPS,transformers,powersupplies,
etc).PowerforITequipmentonly(thedenominator)doesnotincludetheelectricaldistributionlosses
fromUPS,PDU,etconlythepowerprovideddirectlytotheITequipment.
WithPUE,lowerismoreefficient.ThetheoreticallowerlimitofPUEis1.0,butthiswouldnotbe
physicallypossiblesinceitwouldmeanthatnopowerisrequiredforsupportequipment(cooling,
lighting,electricaldistribution,etc).AtypicalPUEisontheorderof1.52.0.BestpracticesPUEsare
nowapproaching1.1(meaningonly100WofpowerforsupportequipmentisrequiredforeachkWof
server/telecomload).ButanyanalysisofPUEshouldtakeintoaccountthatitisadynamic,ever
changingmetricforagivenfacility.Variableserver/telecomloads,ambientweatherconditionsandthe
page27
abilityforcoolingequipmenttoefficientlyrejectheat,andloadfactorsofcoolingandelectrical
distributionequipmentarejustsomeofthefactorsthatcontributetothevariabilityofPUEinagiven
facilityovertime.Duringthedesignprocess,atheoreticalannualaveragePUEcanbedeterminedand
usedasametricfortheefficiencyofthedesign.Thesummaryformthatisapartofthisguidecontainsa
sectioninwhichthedesignteamistoprovidetwoestimatedannualaveragePUEsoneatdayoneof
facilityoperation,andtheotheratfulldesign(server/telecom)capacity.ThedayonePUEcapturesthe
inefficienciesthatarecommonwhencoolingandelectricaldistributionsystemsaresizedforthepeak
load,butthatpeakloadisnotrealizeduntilmonthsoryearsafterthefacilitybeginsoperation.Thefull
designcapacityPUEillustrateshowtheefficiencychangesasthesupportequipmentbecomesfully
loaded.
ThefollowingguidanceisprovidedtocalculateannualaveragePUE(dayoneorfullcapacity):
FanPower:
Fanpower(kW)=
(cfm)*(TSP)*(0.746)
(6345)*(fanmotordrive)
cfm=cubicfeetperminuteofairflow
TSP=totalstaticpressureininchesofwater
=efficiencyoffan,motor,drive,orVFD
PumpPower:
Pumppower(kW)= (gpm)*(TDH)*(0.746)
(3960)*(pumpmotordrive)
gpm=gallonsperminuteofwaterflow
TDH=totaldischargeheadinfeetofwater
=efficiencyofpump,motor,drive,orVFD
CoolingPower:
COP,EER,etccanbeconvertedtokW/ton
Coolingload(tons)multipliedbyefficiency(kW/ton)givescoolingpower(kW)
Forwatercooledchillerplants,besuretoincludecoolingtower(s)
Pumpsarecalculatedseparately(seepreceding)
Ifusingchilledwaterfromcentralutilityplant,useannualaverageplantefficiencyof
0.74kW/tonforcalculations
HumidificationPower:
Firststep:evaluatewhetherhumiditycontrolisnecessary!
Ifhumidificationisdeterminedtobenecessaryeither:
Estimateanannualaveragehumidifierpower,or
Conductanhourlyanalysisforatypicalyear
page28
Witheithermethod,dontforgettoincludelatentcoolingload
(dehumidification)atcoolingplant/system
Centralizethehumiditycontroltoavoidsimultaneoushumidification/dehumidification
oractualenergyusewillbemuchhigherthanthetheoreticalestimate
Lightingpower:
Duringdesign,lightingpowerdensity(LPD)isdeterminedasabasisoflightingdesign
LPDgenerallyexpressedinWattspersquarefoot
EnergycodesdictatemaximumLPDgenerallyintendedforoffice/commercialspaces
andnotexactlyapplicabletodatacentersandtelecomspaces
Besuretoincludeoccupancycontrolprofilesinanycalculationofannuallightingpower
ElectricalLosses:
LossesforUPS,PDU,etccanbedeterminedfrommanufacturersdatabaseduponload
factor,ormeasureddirectlyattheequipment(inputoutput=losses)
CanbeVERYsignificantdependinguponloadfactorsofUPS,PDU,transformers,etc.
Lowerloadsthandesignusuallyresultinhigherelectricallosses(thananticipatedduring
design)
page29

Stanford Server and Telecom Rooms Design Guide - Final - 2011!01!27

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stanford Server and Telecom Rooms Design Guide - Final - 2011!01!27

Uploaded by

Copyright:

Available Formats

StanfordUniversityServer/TelecomRooms

1.25 W/cfm 0.75 W/cfm 0.5 W/cfm

You might also like