Professional Documents
Culture Documents
Power Redundancy in The Data Center Transcript
Power Redundancy in The Data Center Transcript
Transcript
Slide 1
Welcome to the Data Center University™ course: Power Redundancy in the Data Center.
Slide 2: Welcome
For best viewing results, we recommend that you maximize your browser window now. The screen controls
allow you to navigate through the eLearning experience. Using your browser controls may disrupt the
normal play of the course. Click the attachments link to download supplemental information for this course.
Click the Notes tab to read a transcript of the narration.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 4: Introduction
A key element relative to all data centers is the need for power. In most countries, the public power
distribution system is fairly reliable. However, studies have shown that even the best utility systems are
inadequate to meet the strict operating needs of critical nonstop data processing functions. Most companies,
when faced with the likelihood of downtime, and data processing errors caused by faulty utility power
choose to implement a back-up strategy for their mission-critical equipment.
Slide 5: Introduction
These strategies may involve the inclusion of additional hardware such as Uninterruptible Power Supplies
(or UPSs) and generators, and system designs such as N+1 configurations, and dual-corded equipment.
This course will address various strategies to consider when planning for redundancy in the data center.
Slide 6: Introduction
In our rapidly changing global marketplace, the demand for faster, more robust technologies in a smaller
footprint is ever-increasing. In addition, there is a further requirement that these technologies be highly
available as well.
Slide 7: Introduction
Availability is the primary goal of all data centers and networks. Five 9’s of availability of a data center is a
standard most IT professionals strive to achieve. Availability is the estimated percentage of time that
electrical power will be online and functioning properly to support the critical load. It is of critical importance,
and is the foundation upon which successful businesses rely. According to the National Archives and
Records Administration in Washington, D.C., 93% of businesses that have lost availability in their data
center for 10 days or more have filed for bankruptcy within one year. The cost of one episode of downtime
can cripple an organization. The availability of the public power distribution, while sufficient for many
organizations, is ill-equipped to support mission-critical functions. Therefore, planning for redundancy or the
introduction of alternate or additional means of support is a necessity. Redundancy can be thought of as
safety-net or Plan B should power utility fail, or be inadequate. One of the ways in which to increase data
center power availability is through a UPS.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 8: Uninterruptible Power Supplies
An uninterruptible power supply or UPS, in simplistic terms is a device which provides battery back-up
power to IT equipment should utility power be unavailable, or inadequate. UPSs provide power in such a
way that the transition from utility power to battery power is seamless and uninterrupted. UPSs can range in
size and capacity to provide power to small individual desktop computers, all the way up to large megawatt
data centers. Many UPS systems incorporate software management capabilities which allow for data
saving and unattended shutdown should the need or application warrants it.
Understanding how these various UPS designs work is critical to choosing the best UPS for a particular
application.
The Standby UPS Is the most common design configuration used for personal computers. The operating
principal behind the standby UPS is that it contains a transfer switch which by default, uses filtered AC
power as the primary power source. When AC power fails, the UPS switches to the battery by way of the
transfer switch. The battery-to-AC power converter, also known as the inverter is not always on, hence the
name ‘standby.’
The primary benefits of this type of UPS are high efficiency, small in size and low in cost. Some models
are also able to provide adequate noise filtration and surge suppression. The limitations are that this type of
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
UPS uses its battery during brownouts, which degrades overall battery life. Also, it is an impractical solution
over 2kVA.
When AC power fails, the UPS switches to the battery by way of the transfer switch. The battery-to-AC
power converter, also known as the inverter is not always on, hence the name ‘standby.’
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
The primary benefits of this type of UPS are high efficiency, small in size and low in cost. Some models
are also able with proper filter and surge circuitry, to provide adequate noise filtration and surge suppression.
The limitations are that this type of UPS uses its battery during brownouts, which degrades overall battery
life. Also, it is an impractical solution over 2kVA.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
When the input power fails, the transfer switch opens and the power flows from the battery to the UPS
output. With the inverter always on and connected to the output, this design provides additional filtering and
reduces switching transients when compared with the Standby UPS topology.
In addition, the Line Interactive design usually incorporates transformer which adds voltage regulation as the
input voltage varies. Voltage regulation is an important feature when variable voltage conditions exist,
otherwise the UPS would frequently transfer to battery and then eventually down the load. This more
frequent battery usage can cause premature battery failure.
The primary benefits of the Line-interactive UPS topology include high efficiency, small size, low cost and
high reliability. Additionally, the ability to correct low or high line voltage conditions make this the dominant
type of UPS in the 0.5-5kVA power range. The Line-interactive UPS is ideal for rack or distributed servers
and/or harsh power environments. Over 5kVA, the use of a line –interactive UPS becomes impractical.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 12: Standby-Ferro UPS
The Standby-Ferro UPS was once the dominant form of UPS in the 3-15kVA range. This design depends
on a special saturating transformer that has three windings (power connections). The primary power path is
from AC input, through a transfer switch, through the transformer, and to the output.
In the case of a power failure, the transfer switch is opened, and the inverter picks up the output load.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
In the Standby-Ferro design, the inverter is in the standby mode, and is energized when the input power
fails and the transfer switch is opened. The transformer has a special "Ferro-resonant" capability, which
provides limited voltage regulation and output waveform "shaping". The isolation from AC power transients
provided by the Ferro transformer is as good as or better than any filter available. But the Ferro transformer
itself creates severe output voltage distortion and transients, which can be worse than a poor AC connection.
Even though it is a standby UPS by design, the
Standby-Ferro generates a great deal of heat because the Ferro-resonant transformer is inherently
inefficient. These transformers are also large relative to regular isolation transformers; so standby-Ferro
UPS are generally quite large and heavy.
Standby-Ferro UPS systems are frequently represented as On-Line units, even though they have a transfer
switch, the inverter operates in the standby mode, and they exhibit a transfer characteristic during an AC
power failure.
The primary benefit of this design is high reliability and excellent line filtering. The limitations include very
low efficiency combined with instability when used with some generators and newer power-factor corrected
computers, causing the popularity of this design to decrease significantly.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
The Double Conversion On-Line UPS converts AC power to DC and then converts the DC back to AC to
power the connected equipment. The batteries are directly connected to the DC level. This effectively filters
out line noise and all other anomalies from the AC power.
Failure of the AC Power does not cause activation of the transfer switch, because the input AC is charging
the backup battery source which provides power to the output inverter. Therefore, during an AC power
failure, on-line operation results in no transfer time.
There are certainly benefits and limitations of this UPS. A benefit is that it provides nearly ideal electrical
output performance, with no transfer time. But the constant wear on the power components reduces
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
reliability over other designs. Additionally, both the battery charger and the inverter convert the entire load
power flow, resulting in reduced efficiency and increased heat generation. Additionally, the inefficiency of
electricity energy consumption is a significant part of the life-cycle cost of the UPS.
On-Line design, the Delta Conversion On-Line UPS always has the inverter supplying the load voltage.
However, in this configuration the primary power source is blended with power from the additional Delta
Converter. As the primary power varies away from its normal value the inverter comes to life to make up the
difference. The Double Conversion On-Line UPS converts the power to the battery and back again whereas
the Delta Converter moves components of the power from input to the output.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
In the Delta Conversion On-Line design, the Delta Converter acts with dual purposes. The first is to control
the input power characteristics. This active front end draws power in a sinusoidal manner, minimizing
harmonics reflected onto the utility. This ensures optimal utility and generator system compatibility,
reducing heating and system wear in the power distribution system. The second function of the Delta
Converter is to control input current in order to regulate charging of the battery system.
This input power control makes the Delta Conversion On-Line UPS compatible with generators and reduces
the need for wiring and generator over sizing. Delta Conversion On-Line technology is the only core UPS
technology today protected by patents and is therefore not likely to be available from a broad range of UPS
suppliers.
The benefits of Delta Conversion On-Line UPS include high efficiency, excellent voltage regulation, and
overall reduction in life-cycle costs of energy in large installations. It is impractical in installations under
5kVA.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
1. Capacity or “N” System
2. Isolated Redundant
3. Parallel Redundant or “N+1” System
4. Distributed Redundant
5. System plus System Redundant
Before we can fully explain these redundancy configurations, we must first talk about the concept of ‘N.’
On the other hand, if there are 5 disks and only 4 are needed for storage capacity, this is an example of an
N+1 design.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
If 4 disks are required for storage capacity, and there are 2 RAID systems, each with 4 disks, this is
considered 2N.
Occasionally, N+2 designs are seen, but often, this are happened upon by mistake, rather than a purposeful
design.
For example, a 6 disk RAID may be utilized with the thought that that much storage capacity would be used,
however, the capacity never grows larger than the capacity of 4 disks. In this case, this is actually a N+2
design. N+1, and 2N designs are those that have some form of built in redundancy.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 18: Capacity or “N” System
Now let’s get back to UPS Redundancy designs. The first is the Capacity or “N” System. An N system
simply stated is a system comprised of a single UPS, or a paralleled set of UPSs whose capacity is equal to
the load.
This type of system is by far the most common of the configurations in the UPS industry. The small UPS
under an office desk protecting one desktop is an N configuration. Likewise, a very large 400 kW computer
room is an N configuration whether it has a single 400 kW UPS, or two paralleled 200 kW UPSs. An N
configuration can be looked at as the minimum requirement to provide protection for the critical load.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Optimal efficiency of the UPS, because the UPS is used to full capacity
Provides availability over that of the utility power
Expandable if the power requirement grows (It is possible to configure multiple units in the same
installation. Depending on the vendor or manufacturer, you can have up to 8 UPS modules of the
same rating in parallel.)
Disadvantages:
Limited availability in the event of a UPS module break down, as the load will be transferred to
bypass operation, exposing it to unprotected power
During maintenance of the UPS, batteries or down-stream equipment, load is exposed to
unprotected power (usually takes place at least once a year with a typical duration of 2-4 hours)
Lack of redundancy limits the load’s protection against UPS failures
Many single points of failure, which means the system is only as reliable as its weakest point
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
In a normal operating scenario the primary UPS module will be carrying the full critical load, and the
isolation module will be completely unloaded. Upon any event where the primary module(s) load is
transferred to static bypass, the isolation module would accept the full load of the primary module
instantaneously. The isolation module has to be chosen carefully to ensure that it is capable of assuming
the load this rapidly. If it is not, it may, itself, transfer to static bypass and thus defeat the additional
protection provided by this configuration.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Relatively cost effective for a two-module system
Disadvantages:
Reliance on the proper operation of the primary module's static bypass to receive power from the
reserve module
Requires that both UPS modules’ static bypass must operate properly to supply currents in excess
of the inverter's capability
Complex and costly switchgear and associated controls
Higher operating cost due to a 0% load on the secondary UPS, which draws power to keep it
running
A two module system (one primary, one secondary) requires at least one additional circuit breaker
to permit choosing between the utility and the other UPS as the bypass source. This is more
complex than a system with a common load bus and further increases the risk of human error.
Two or more primary modules need a special circuit to enable selection of the reserve module or
the utility as the bypass source (Static Transfer Switch)
Single load bus per system, a single point of failure
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
The system is N+1 redundant if the “spare” amount of power is at least equal to the capacity of one system
module; the system would be N+2 redundant if the spare power is equal to two system modules; and so on.
Parallel redundant systems require UPS modules of the same capacity from the same manufacturer. The
UPS module manufacturer also provides the paralleling board for the system. The paralleling board may
contain logic that communicates with the individual UPS modules, and the UPS modules will communicate
with each other to create an output voltage that is completely synchronized. The number of UPS modules
that can be paralleled onto a common bus is different for different UPS manufacturers. The UPS modules in
a parallel redundant design share the critical load evenly in normal operating situations. When one of the
modules is removed from the parallel bus for service (or if it were to remove itself due to an internal failure),
the remaining UPS modules are required to immediately accept the load of the failed UPS module. This
capability allows any one module to be removed from the bus and be repaired without requiring the critical
load to be connected to straight utility.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Advantages:
Higher level of availability than capacity configurations because of the extra capacity that can be
utilized if one of the UPS modules breaks down
Lower probability of failure compared to isolated redundant because there are less breakers and
because modules are online all the time (no step loads)
Expandable if the power requirement grows. It is possible to configure multiple units in the same
installation
The hardware arrangement is conceptually simple, and cost effective
Disadvantages:
Both modules must be of the same design, same manufacturer, same rating, same technology and
configuration
Still single points of failure upstream and downstream of the UPS system
The load is exposed to unprotected power during maintenance of the UPS, batteries or down-
stream equipment, which usually takes place at least once a year with a typical duration of 2-4
hours
Lower operating efficiencies because no single unit is being utilized 100%
Single load bus per system, a single point of failure
Most manufacturers need external static switches in order to load-share equally between the two
UPS modules; otherwise they will share within a wide window of 15%. This adds to the cost of the
equipment and makes it more complex
Most manufacturers need a common external service bypass panel. This adds to the cost of the
equipment and makes it more complex
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
This design was developed in the late 1990s in an effort by an engineering firm to provide the capabilities of
complete redundancy without the cost associated with achieving it. The basis of this design uses three or
more UPS modules with independent input and output feeders. The independent output buses are
connected to the critical load via multiple Power Distribution Units and Static Transfer Switches. From the
utility service entrance to the UPS, a distributed redundant design and a system plus system design
(discussed in the next section) are quite similar. Both provide for concurrent maintenance, and minimize
single points of failure. The major difference is in the quantity of UPS modules that are required in order to
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
provide redundant power paths to the critical load, and the organization of the distribution from the UPS to
the critical load. As the load requirement, “N”, grows the savings in quantity of UPS modules also increases.
Distributed redundant systems are usually chosen for large complex installations where concurrent
maintenance is a requirement and many or most loads are single corded. Savings over 2N also drive this
configuration.
Advantages:
Allows for concurrent maintenance of all components if all loads are dual-corded
Cost savings versus a 2(N+1) design due to fewer UPS modules
Two separate power paths from any given dual-corded load’s perspective provide redundancy from
the service entrance
UPS modules, switchgear, and other distribution equipment can be maintained without transferring
the load to bypass mode, which would expose the load to unconditioned power. Many distributed
redundant designs do not have a maintenance bypass circuit.
Disadvantages:
Relatively high cost solution due to the extensive use of switchgear compared to previous
configurations
Design relies on the proper operation of the STS equipment which represents single points of
failure and complex failure modes
Complex configuration; In large installations that have many UPS modules and many static transfer
switches and PDUs, it can become a management challenge to keep systems evenly loaded and
know which systems are feeding which loads.
Unexpected operating modes: the system has many operating modes and many possible
transitions between them. It is difficult to test all of these modes under anticipated and fault
conditions to verify the proper operation of the control strategy and of the fault clearing devices.
UPS inefficiencies exist due to less than full load normal operation
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 22: System Plus System Redundant or “2N”
System plus System, Multiple Parallel Bus, Double-Ended, 2(N+1), 2N+2, [(N+1) + (N+1)], and 2N are all
nomenclatures that refer to variations of this configuration.
With this design, it now becomes possible to create UPS systems that may never require the load to be
transferred to the utility power source. These systems can be designed to exhaust every conceivable single
point of failure. However, the more single points of failure that are eliminated, the more expensive this
design will cost to implement. Most large system plus system installations are located in standalone,
specially designed buildings. It is not uncommon for the infrastructure support spaces (UPS, battery,
cooling, generator, utility, and electrical distribution rooms) to be equal in size to the data center equipment
space.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
This is the most reliable, and most expensive, design in the industry. It can be very simple or very complex
depending on the engineer’s vision and the requirements of the owner. Although a name has been given to
this configuration, the details of the design can vary greatly and this, again, is in the vision and knowledge of
the design engineer responsible for the job. The 2(N+1) variation of this configuration revolves around the
duplication of parallel redundant UPS systems. Optimally these UPS systems would be fed from separate
switchboards, and even from separate utility services and possibly separate generator systems. The
extreme cost of building this type of facility has been justified by the importance of what is happening within
the walls of the data center and the cost of downtime to operations. Many of the world’s largest
organizations have chosen this configuration to protect their critical load.
The fundamental concept behind this configuration requires that each piece of electrical equipment can fail
or be turned off manually without requiring that the critical load be transferred to utility power. Common in
2(N+1) design are bypass circuits that will allow sections of the system to be shut down and bypassed to an
alternate source that will maintain the redundant integrity of the installation.
For example if a critical load is 300 kW, the design requires that four 300 kW UPS modules be provided, two
each on two separate parallel buses. Each bus feeds the necessary distribution to feed two separate paths
directly to the dual-corded loads. The single-corded load, illustrated in Figure 6, shows how a transfer
switch can bring redundancy close to the load. However, Tier IV power architectures require that all loads
to be dual-corded.
Companies that choose system plus system configurations are generally more concerned about high
availability then the cost of achieving it. These companies also have a large percentage of dual-corded
loads.
Advantages:
Two separate power paths allows for no single points of failure; Very fault tolerant
The configuration offers complete redundancy from the service entrance all the way to the critical
loads
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
In 2(N+1) designs, UPS redundancy still exists, even during concurrent maintenance
UPS modules, switchgear, and other distribution equipment can be maintained without transferring
the load to bypass mode, which would expose the load to unconditioned power
Easier to keep systems evenly loaded and know which systems are feeding which loads.
Disadvantages:
Highest cost solution due to the amount of redundant components
UPS inefficiencies exist due to less than full load normal operation
Typical buildings are not well suited for large highly available system plus system installations that
require compartmentalizing of redundant components
The second factor is Risk Tolerance. Companies that have not experienced a major failure are typically
more risk tolerant than companies that have not. Smart companies will learn from what companies in their
industry are doing. This is called “Benchmarking” and it can be done in many ways. The more risk
intolerant a company is, the more internal drive their will be to have more reliable operations, and disaster
recovery capabilities.
Other factors to consider are availability requirements – How much downtime can the company withstand in
a typical year? If the answer is none, then a high availability design should be in the budget. However, if
the business can shut down every night after 10 PM, and on most weekends, then the UPS configuration
wouldn’t need to go far beyond a parallel redundant design. Every UPS will, at some point, need
maintenance, and UPS systems do fail periodically, and somewhat unpredictably. The less time that can be
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
found in a yearly schedule to allow for maintenance the more a system needs the elements of a redundant
design.
Types of loads (single vs. dual-corded) are another area to examine. Dual-corded loads provide a real
opportunity for a design to leverage a redundant capability, but the system plus system design concept was
created before dual-corded equipment existed. The computer manufacturing industry was definitely
listening to their clients when they started making dual-corded loads. The nature of loads within the data
center will help guide a design effort, but are much less a driving force than the issues stated above.
Finally, budget. The cost of implementing a 2(N+1) design is significantly more, in every respect, than a
capacity design, a parallel redundant design, or even a distributed redundant. As an example of the cost
difference in a large data center, a 2(N+1) design may require thirty 800 kW modules (five modules per
parallel bus; six parallel busses). A distributed redundant design for this same facility requires only eighteen
800 kW modules, a huge cost savings.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Most of this excess cost can be recovered by implementing a method and architecture that can adapt to
changing requirements in a cost-effective manner while at the same time providing high availability.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 27: Dual Path Environments
However, equipment with a single power path (single-corded) introduces a weakness into an otherwise
highly available data center. Transfer switches are often used to enhance single-corded equipment
availability by providing the benefits of redundant utility paths. A transfer switch is a common component in
data centers and is used to perform the following functions:
1. Switching UPS and other loads from utility to generator during a utility power failure
2. Switching from a failed UPS module to utility or another UPS (depending on designs)
3. Switching critical IT loads from one UPS output bus to another in a dual path power system
2. Use a transfer switch at the point of use to select a preferred source, and when that source fails
switch to the second power path
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
3. Use a large centralized transfer switch fed from the two sources, to generate a new power bus to
supply a large group of single corded loads
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
The ATS is fed by two sources, the utility and the generator, with the utility the preferred source. When the
preferred source is unacceptable, the ATS automatically switches to the generator. Standby generator
systems are often used in conjunction with UPS systems.
© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.