HP-3PAR

PERFORMANCE PRESENTATION

October 2010

© Copyright 2010 Hewlett-Packard Development Company, L.P.

The information contained herein is subject to change without notice. HP Confidential
COURSE OBJECTIVES
– At the end of this presentation the student should be able to:
• Understand Individual Component Performance
• Describe RAID impact to overall performance
• Understand How 3PAR caching works
• Setup FC Host Port Considerations for proper balancing
• Use published performance data you can use in sales situations

2 HP Confidential | October 2010

COMPONENT PERFORMANCE
Assumptions
All performance numbers are using the following load:

IOP/s
– Small block size (16k or smaller)
• There is little difference between IOPs that are 16K or smaller.
– Random access to entire device
– Multiple threads to the device

MB/s (Throughput)
– Large block size (256k or larger)
– Sequential access to entire device
– Single thread to the device

3 HP Confidential | October 2010

COMPONENT PERFORMANCE
DI Disk Type/Speed IOP/s MB/s
SK 15K FC 200 45
10K FC 150 45
7.2K NL 75 30

Worlkload IOPs Aligned IOPs Unaligned

100% Read 3950 3900
SS 70% Read/30% Write 2100 1500
D 50% Read/50% Write 1800 1150
30% Read/70% Write 1500 1000
100% Write 1600 1150

NOTES:
1. These are back end numbers. RAID overhead must be considered when calculating front end capability
2. Numbers reflect IO access from VLUN (host) to Physical Disk
3. IOP/s are less with larger blocks! (With 64k blocks, IOP/s are 67% of above)
4. As seen above, SSD IOPs vary greatly with IO Mix
5. SSD IOPs above are 4K. Larger blocks has impact on overall SSD performance
6. SSD writes are significantly slower than reads (2 – 3 times longer to complete)
7. SSD Sequential performance is slower than spinning disks

4 HP Confidential | October 2010

COMPONENT PERFORMANCE

FC Ports and InServ Cages

Bandwidth IOP/s
4Gb Port 360 MB/s 30,000
15K FC = 8000
Drive Chassis 720 MB/s 10K FC = 6000
7.2K NL = 3000

5 HP Confidential | October 2010

COMPONENT PERFORMANCE
Node-Pairs (RAID 1, Host/Disk)
Reads MB/s Writes MB/s Reads IOPs Writes IOPs
T-Series 1400 / 1400 600 / 1200 64k / 64k 32k / 64k
F-Series 1300 / 1300 500 / 1000 38.4k / 38.4k 19.2k / 38.4k

NOTES:
1. Appropriate number of disks and HBA’s needed to obtain node max
2. Numbers above are listed for a node-pair (2 nodes)
3. Disk IOP/s are limited by drive support (max config)

6 HP Confidential | October 2010

COMPONENT PERFORMANCE
Node-Pairs (RAID 5, Host/Disk)
Reads MB/s Writes MB/s Reads IOPs Writes IOPs
T-Series 1400 / 1400 560 / 750 64k / 64k 16k / 64k
F-Series 1300 / 1300 470 / 625 38.4k / 38.4k 9.6k / 38.4k

NOTES:
1. Appropriate number of disks and HBA’s needed to obtain node max
2. Numbers above are listed for a node-pair (2 nodes)
3. Disk IOP/s are limited by drive support (max config)
4. Host Write MB/s depend on set size (shown as default of 4 for R5, 3d+1p)
5. Backend IOP performance is fixed per node and front end depends on data to parity overhead. In this example
750MB/sec back end is 560 for host data and 190MB/sec for parity. If this was 7+1 the host could push 650 MB/s

7 HP Confidential | October 2010

COMPONENT PERFORMANCE
Node-Pairs (RAID 6, Host/Disk)
Reads MB/s Writes MB/s Reads IOPs Writes IOPs
T-Series 1400 / 1400 560 / 750 64k / 64k 9.6k / 64k
F-Series 1300 / 1300 470 / 625 38.4k / 38.4k 5.8k / 38.4k

NOTES:
1. Appropriate number of disks and HBA’s needed to obtain node max
2. Numbers above are listed for a node-pair
3. For all other Series, disk IOP/s are limited by drive support (max config)
4. Host Write MB/s depend on set size (shown as default of 8 for R6, 6d+2p)
5. Host Write MB/s equation is [(set size - 2) / (set size)] * [Disk Bandwidth]

8 HP Confidential | October 2010

WHY DO R6 WRITES HAVE A 6.66 OVERHEAD?
– RAID-5 has a 4 IO back end penalty. As a write comes in the old Parity and old data
must be read (two read IOPs) then parity calculated and new parity and new data
written out (two write IOPs). So each random write to R5 has 4 back end IOPs (2 read
+ 2 write)
– RAID-6 calculation uses the XOR engine in the ASIC but must calculate two distinct
parities (R5 only needs one parity) and need to compute parities over more data
– Most writes require only two parities. However, a fraction (1/3 for step size 8), of the
data blocks are used to compute 3 parities, so updating those blocks requires
reading/updating 3 parities, hence the odd number 6.66 back end IOPs for R6

9 HP Confidential | October 2010

HIGH LEVEL CACHING ALGORITHM (1 OF 2)
– Cache is mirrored across nodes to ensure there won’t be data loss in the event
of a node failure.
– The algorithm is self-adapting, determining in real-time the allocation for
writes versus reads.
• Under intensive reads, up to 100% cache per node may be used.
• Under intensive writes, up to 50% cache per node may be used.

– The Inserv is not cache centric like monolithic architectures. Cache is used
primarily as a buffer to facilitate data movement to and from disk.

10 HP Confidential | October 2010

HIGH LEVEL CACHING ALGORITHM (2 OF 2)
– We have an intelligent flusher algorithm which flushes the dirty cache data out
to disk at a rate that maximizes cache hits yet ensures there are adequate free
pages for new host writes.
• Advantages:
− Merging multiple writes to the same page.
− Coalescing small block writes into large block writes.
− Combining multiple R5 and R6 writes into full stripe writes.

– For spinning disks the Inserv will read a full 16K cache page from disk.
• If a host read is for less than 16K, we still read in the entire page.
• If a write is for less than a full page, we only write out the partial page with valid data.
• We will combine multiple sub-page size host writes in a single dirty page.

11 HP Confidential | October 2010

READ-AHEAD ALGORITHM (1 OF 2)
– Goal: detect sequential reads so they become mostly cache resident and hence
much faster
– We detect and pre-fetch for up to 5 interlaced streams per virtual volume
– Streams do not need to be purely sequential
• Our pre-fetch algorithm is adaptive and will trigger when multiple pages in a set range or zone
are read. This allow us to read ahead on various patterns of reads such as reading every other
block

12 HP Confidential | October 2010

READ-AHEAD ALGORITHM (2 OF 2)
– Specifically, when an I/O comes in, a zone is computed that is equal to 8 times
the I/O size. For example, an 8KB I/O will have a 64KB zone. If three reads
hit in this zone a read-ahead is triggered. The read ahead is 1MB up to 4MB
depending on the incoming I/O size.
– If the I/O size from the host is 512KB or less, then the read-ahead size is
iosize*8 with a minimum of 1MB.
– If the iosize from the host is greater than 512KB then the read-ahead size is
iosize*4 with a maximum of 4MB.

13 HP Confidential | October 2010

PUBLISHED 3PAR BENCHMARK RESULTS
– SPC-1 (Storage Performance Council)
• Results at: www.storageperformance.org/results/benchmark_results_spc1/
• T800 InServ (September 2008)
− 224,989 SPC-1 IOPS with 7.22 ms response time
− 77,824 GB user size (ASU)
− 1280 x 146GB 15K FC drives
• F400 InServ (April 2009)
− 93,050 SPC-1 IOPS with 8.85 ms response time
− 27,046 GB user size (ASU)
− 384 x 146GB 15K FC drives

14 HP Confidential | October 2010

PUBLISHED 3PAR BENCHMARK RESULTS
– ESRP (Microsoft Exchange Solution Reviewed Program)
• Results at: http://technet.microsoft.com/en-us/exchange/bb412164.aspx
•5 submissions for Exchange 2007 in Spring 2009
• T800 (4-node) InServ (April 2009)
− 96,000 Exchange Mailboxes (300 MB each)
− 0.576 IOPS/Mailbox (very heavy)
− 640 x 146GB 15K FC drives

15 HP Confidential | October 2010

THANK YOU

16 HP Confidential | October 2010

BENCHMARKS CONFIGURATION
– Information to Gather:
• Workload: block size, R/W ratio, threads per device, access pattern
• Volumes: size, count, raid type, availability, snapspace, provisioning
• Hosts: architecture, OS, HBA’s, drivers, MP, topology, etc.
– InServ Volumes/Settings:
• Maximize chunklet use on PD’s and use all PD’s evenly
• Port balancing (front-end AND back-end)
• Volume layout vs. Host layout (on node pairs)

17 HP Confidential | October 2010

BENCHMARKS CONFIGURATION – LOAD
GENERATORS
– Stress – 3PAR Internal Tool
• http://engweb.3pardata.com/software/stressd.html
• Pros: Easy to use, heavy load, low memory use
• Cons: Not a public tool

– IOMeter – Measurement / Characterization Tool

• http://www.iometer.org/
• Pros: Lots of load and profiling options
• Cons: Only available on Windows, lots of options

– Vdbench – Data Integrity and Performance Tool

• http://sourceforge.net/projects/vdbench/
• Pros: Can specify I/O rate instead of just threads
• Cons: Java based, can get CPU intensive

18 HP Confidential | October 2010

BENCHMARKS CONFIGURATION – LOAD
GENERATORS
– IOzone – File System Performance Tool
• http://www.iozone.org/
• Pros: Provides a broad file system analysis
• Cons: This is a file system benchmark
− Performance of this test can depend on host memory
− File system load changes before hitting storage
− Has hundreds of possible tests. Depending on how run, one test can still be running (to array) during next
test
– JetStress – Exchange Performance Tool
• http://www.microsoft.com/downloads/en/
• Pros: Simulate load on a server running Exchange to verify the performance and stability of disk
subsystem
• Cons: Can only be run on a test server, not production

19 HP Confidential | October 2010

FC HOST PORT CONSIDERATIONS
– Initiators per FC Host Port
• Per Port: 64
• Per InServ: 1024

– Queue Depth per FC Host HBA

• 4Gb: 1638

20 HP Confidential | October 2010

PERFORMANCE EQUATIONS
– You need to know:
•R = Read Percent (50% = 0.50)
• W = Write Percent (50% = 0.50)
• FE = Front End IOPS
• BE = Back End IOPS

– You can answer questions such as:

1. How many nodes/drives/cages do I need to do 20,000 8K, 30% write IOPS?
2. How many R/W IOPS can I get out of 2 nodes, 4 drive chassis and 80 FC 15K drives?
How much throughput?

21 HP Confidential | October 2010

PERFORMANCE EQUATIONS – RAID1
R1: Base VV
FE * (R + 2W) = BE
(0 Base R, 2 Base W)

R1: CPVV w/R1 SS

FE * (R + 5W) = BE
(1 Base R, 2 Base W, 0 SD R, 2 SD W)

R1: CPVV w/R5 SS

FE * (R + 7W) = BE
(1 Base R, 2 Base W, 2 SD R, 2 SD W)

R1: CPVV w/R6 SS

FE * (R + 9.66W) = BE
(1 Base R, 2 Base W, 3.33 SD R, 3.33 SD W)

R1: TPVV
FE * [R + (>3W)] = BE
(0 Base R, 2 Base W, 1+ SA W)

22 HP Confidential | October 2010

PERFORMANCE EQUATIONS – RAID5
R5: Base VV
FE * (R + 4W) = BE
(2 Base R, 2 Base W)

R5: CPVV w/R1 SS

FE * (R + 7W) = BE
(3 Base R, 2 Base W, 0 SD R, 2 SD W)

R5: CPVV w/R5 SS

FE * (R + 9W) = BE
(3 Base R, 2 Base W, 2 SD R, 2 SD W)

R5: CPVV w/R6 SS

FE * (R + 11.66W) = BE
(3 Base R, 2 Base W, 3.33 SD R, 3.33 SD W)

R5: TPVV
FE * [R + (>5W)] = BE
(2 Base R, 2 Base W, 1+ SA W)

23 HP Confidential | October 2010

PERFORMANCE EQUATIONS – RAID6
R6: Base VV
FE * (R + 6.66W) = BE
(3.33 Base R, 3.33 Base W)

R6: CPVV w/R1 SS

FE * (R + 9.66W) = BE
(4.33 Base R, 3.33 Base W, 0 SD R, 2 SD W)

R6: CPVV w/R5 SS

FE * (R + 11.66W) = BE
(4.33 Base R, 3.33 Base W, 2 SD R, 2 SD W)

R6: CPVV w/R6 SS

FE * (R + 14.33W) = BE
(4.33 Base R, 3.33 Base W, 3.33 SD R, 3.33 SD W)

R6: TPVV
FE * [R + (>7.66W)] = BE
(3.33 Base R, 3.33 Base W, 1+ SA W)

24 HP Confidential | October 2010

CACHE MEMORY PERFORMANCE (1 OF 2)
– Top Table – Per node Read/Write Cache hit statistics:
• Accesses – Number of Current and Total Read/Write I/Os
• Hits (Read) – Number of read I/Os in which data was already in cache
• Hits (Write) – Number of write I/Os in which data was already in cache
• Hit% - Hits divided accesses displayed in percentage

25 HP Confidential | October 2010

CACHE MEMORY PERFORMANCE (2 OF 2)
– Page Stats:
• Free – Number of cache pages without valid data on them
• Clean – Number of clean cache pages (valid data on page)
− A page is clean when data in cache matches data on disk
• Write 1 – Number of dirty pages that have been modified exactly 1 time
− A page is dirty when it has been modified in cache but not written to disk
• Write N – Number of dirty pages that have been modified more than 1 time

26 HP Confidential | October 2010

STAT COMMANDS THAT ARE USEFUL

Front-End Stat Commands Node Stat Commands

•statvlun -ni –rw •statcmp
•statport -host -ni –rw •statcpu

Back-End Stat Commands RC/iSCSI Commands

•statpd -ni -rw •statiscsi
•statport -disk -ni –rw •statiscsisession
•statrcopy
•statport –rcip/rcfc

NOTE: These should be collected in parallel, along with host stats!

27 HP Confidential | October 2010