WHITE PAPER
CREATING STRETCH CLUSTERS
WITH STORMAGIC SvSAN
Overview
Many physical disruptions and disasters that
organizations face are localized to a specific
room, building or campus. Much maintenance
and repair work that organizations undertake
is similarly contained to a single location or is
completed in a staged manner. For IT operations,
these more-frequent, localized events cause
significant disruption, but the disruption can
be minimized or eliminated through the use of
stretch clusters.
This white paper will examine the use cases
for stretch clusters, and how StorMagic
SvSAN enables stretch clusters to be created.
Supporting this are test results demonstrating
the level of latency (and therefore distance)
that SvSAN can tolerate for both read and write
operations.
Key considerations in operational
resiliency
1. Localized disruptions and disasters are more
frequent than regional disasters
2. Recovery from many regional disasters can
be accomplished locally without resorting to
a disaster declaration
3. Stretch clusters across buildings, campuses
and metro-areas are a valuable tool in
reducing or eliminating service interruptions
from both local and regional disasters
4. Stretch clusters are relatively low cost,
especially when compared to the cost of an
out-of-region disaster recovery center
5. Stretch clusters help avoid the high cost of
disaster declarations
StorMagic. Copyright © 2018. All rights reserved.
Regional disasters are rare, but
localized disruptions are frequent
Even in high-risk areas, such as California
for earthquakes, Kansas and Oklahoma for
tornadoes, and Louisiana for hurricanes, regional
disasters of sufficient impact to disrupt IT
operations are a relatively rare occurrence.
Smaller disasters and disruptions, however, are
significantly more frequent and have greater
impact on daily operations than the regional
disasters against which organizations protect
themselves. Small disasters may include
localized flooding, fires, air contamination,
bombs from localized terrorist attacks,
and lock-downs due to active-shooter
incidents. Small disruptions may occur
from equipment maintenance, utility
system upgrades, and building
remodeling.
These situations rarely require
a disaster declaration, which
would invoke costly recovery
procedures at a distant
disaster recovery site, but
rather, can typically be
managed by failing
over applications
to another system
located within the
building, campus,
or metropolitan
area. This
approach is
dramatically
less
disruptive
 and, with When the network performance is poor, a
The recommended maximum
distance two SvSAN nodes can be today’s typical response to solve the issue is to increase
stretched across is: stretch network bandwidth, which is a relatively simple
cluster thing to achieve. However, this only improves
1,000Km technology, can
be done quite
performance when there is network congestion.
Bandwidth addresses the amount of data
That's equivalent to the cost effectively. that can be transmitted, but does not govern
distance from New York the speed of the link. In general having more
City to Cincinnati. Creating stretch bandwidth reduces the likelihood of congestion.
clusters with Bandwidth is similar to lanes on a highway –
Expressed as latency,
more lanes enable more vehicles (data packets)
SvSAN can tolerate StorMagic SvSAN
a recommended: to use the highway at the same time. When all
The majority of StorMagic
lanes are full the bandwidth limit is reached,
10ms
SvSAN cluster deployments
which leads to congestion (traffic jams).
will reside in a single physical
location, either in the same
Latency or round trip time (RTT) is another
rack, equipment room or
important factor and determines the speed
site, with cluster nodes usually
of the link – lower latency equates to better
separated by a relatively short
network speeds. Referring back to the highway
distance. Despite this, SvSAN offers
analogy, latency is the highway speed limit and
the storage architect the ability
impacts the time it takes to complete a round
to separate the cluster nodes over
trip. Unfortunately, getting better network latency
greater distances while maintaining
is not as simple as increasing bandwidth as it is
synchronous data replication, thereby
affected by a number of factors, most of which
creating a stretch cluster configuration,
are out of the user’s control. These include:
shown in fig. 1.
• Propagation delay - the time that data takes
Bandwidth and latency - the to travel through a medium relative to the
limitations of using stretch clusters speed of light, such as fibre optic cable, or
If you are considering stretch clusters as part copper wire.
of your storage infrastructure, then you will no
doubt be aware of the bandwidth and latency • Routing and switching delays - the number
limitations that will influence your deployment. of routers and switches data has to pass
through.
Site A Site B
• Data protocol conversions - decoding and re-
encoding slows down data.

• Network congestion and queuing -

bottlenecks caused by routers and switches.

• Application latency - some applications can

introduce or only tolerate a certain amount of
latency.

Putting SvSAN's latency tolerance to

Fig. 1: A typical stretch cluster, with one node
on Site A, another on Site B, and a witness
node separated remotely.
the test
StorMagic has conducted tests with SvSAN
to determine the bandwidth and latency (and
distance) that two nodes within a cluster can
tolerate before service is adversely affected.

StorMagic. Copyright © 2018. All rights reserved.

 Server configuration
The tests were performed using a pair of
Supermicro X7DVL-3 servers with the following
configuration:

Processor 1 x Intel Xeon E5405 4-core 2GHz

Memory 8GB RAM
Network 2 x Intel 80003ES2 Gigabit Ethernet
Adapter Adapters

RAID 1 x LSI MegaRAID 1078 RAID

Controller Controller

Disk Drives 3 x Western Digital 500GB 7.2k RPM

SATA (WD5002ABYS) in a RAID-0
stripe and presented as Raw Device
Mapped (RDM)
Hypervisor VMware vSphere ESXi 5.5 Fig. 3: Graph depicting the impact of latency
on read and write IOPS and response time
I/O load generation software – IOMeter when using SvSAN.
To ensure that the tests were repeatable, I/O load The configuration used for the tests is illustrated
was generated using the IOMeter tool (www. in fig. 2.
iometer.org)
The response times and IOPS remained
The following I/O patterns were used: relatively constant up to latencies of 30ms,
4KB block size, 100% random read after which the random write IOPS rapidly
4KB block size, 100% random writes dropped off and write response times began
8KB block size, 100% random read to rise. These results are shown in fig. 3.
8KB block size, 100% random writes
Disk queue depth = 32 NOTE: Using different disk types (SSDs,
15K RPM, SAS, wider disk stripes,
Latency emulator – WANem etc.) would increase the number of
To simulate different network latencies, the IOPS and/or lower the response
WANem (http://wanem.sourceforge.net) times, but would still follow similar
Wide Area Network Emulator tool was used. profiles.
WANem can be used to simulate Wide Area
Network conditions, allowing different network Additional findings included:
characteristics, such as network delay (latency),
bandwidth, packet loss, packet corruption, • Random read IOPS and
disconnections, packet re-ordering, jitter, etc. to response times are not
be configured. affected by increased
latencies and
remained relatively
constant at 700
IOPS and 45.6ms
through out the
test. The read
requests are
fulfilled by
the local
Fig. 2: Stretch cluster testing configuration. storage,
and

StorMagic. Copyright © 2018. All rights reserved.

 wouldn’t need to traverse the network
• Write IOPS were between 730 and 770 until
the RTT reached 30ms, after which the
number of IOPS rapidly dropped off. This
is because synchronous mirroring requires
all writes to be committed to both sides of
the mirror before being acknowledged as
complete and would have had to traverse the
network
• The write response time averaged ~42.5ms
until the RTT reached 30ms, after which it
began to rise, on reaching 100ms of delay
it increased exponentially. These response
times could lead to unacceptable application
performance.
Recommendations
Should you choose to leverage SvSAN's
stretch cluster feature, and to ensure data
is synchronously replicated, the following
recommendations should be followed:
• Although SvSAN synchronous mirroring
can tolerate high round trip times, it is
recommended that it should not exceed
10ms, equating to 1,000Km excluding other
factors that affect network latency. Latencies
greater than this could lead to performance
degradation or unpredictable behavior.
• Bandwidth between SvSAN VSAs should be a
minimum of 1Gb, this depends on the amount
of data that needs to be synchronously
mirrored.
• Ideally there should be a minimum of
two separate network connections for
redundancy.
• Make certain the path selection policy
ensures that the data is read from the local
side of the mirror to avoid read requests from
traversing the network.
Conclusions
Separating or stretching the nodes within
a storage cluster is a very effective way of
increasing the resiliency of storage infrastructure.
Creating a stretch cluster by placing the nodes
StorMagic. Copyright © 2018. All rights reserved.
in separate racks, rooms, buildings or cities can
almost eliminate downtime associated with
localized disasters such as fire, flooding or power
outages. The versatility of SvSAN's strech cluster
feature enables storage infrastructure to be
configured in this highly resilient manner.
The testing conducted for this white paper
has shown that SvSAN synchronous mirroring
can withstand high latencies allowing the
SvSAN cluster nodes to be separated by up
to 1,000Km, which provides a huge scope for
building resiliency into an organization's storage
infrastructure.
Further Reading
Enabling stretch clusters is just one feature of
many within SvSAN. Why not explore some of
the others, such as Predictive Storage Caching,
or Data Encryption? These features and
more can be accessed through the extensive
collection of white papers on the StorMagic
website.
Additional details on SvSAN are available
in the Technical Overview which details
SvSAN's capabilities and deployment
options.
If you're ready to test SvSAN in your
environment, you can do so totally
free of charge, with no obligations.
Simply download our fully-
functioning free trial of SvSAN
from the website.
If you still have questions, or
you'd like a demo of SvSAN
you can contact the
StorMagic team directly
by sending an email to
sales@stormagic.com StorMagic
Unit 4, Eastgate
Office Centre
Eastgate Road
Bristol
BS5 6XX
United Kingdom
+44 (0) 117 952 7396
sales@stormagic.com
www.stormagic.com