DS8000 Logical Configuration 03152011

Advanced Technical Skills (ATS) North America
DS8000: Storage Configuration and Best Practices
Charlie Burger
Rosemary McCutchen
Hank Sautter
3/11/2011
2010 IBM Corporation
ATS Education - Customers

Accelerate with ATS Education
https://www.ibm.com/developerworks/mydeveloperworks/blogs/accelerate/?order=desc&maxresults=100&sortby=0&lang=en
Previous Events:

Tape Encryption with Tivoli Key Lifecycle Manager (TKLM)

IBM System Storage TS3500 Tape Library Update
Whats new in the DS3000 Disk Family
Proactive Performance Management with TPC
DS8700 Easy Tier
TPC 4.1.1 Performance Management enhancements and

demonstration
Ten Things for the new TPC Administrator to do to make TPC

4.1.1 more useful
Storage Subsystem Performance, Monitoring and Capacity

Planning
XIV Asynchronous Mirror

2
Topics
DS8000 Subsystem Storage Hierarchy
Logical Configuration Considerations
Defining Extent Pools on DS8000
Volume Virtualization Overview
Addendum
Logical Configuration of Volumes Using DS8000 Storage Manager
Logical Configuration of CKD Volume Using DS CLI
DS8000 Volumes with z/OS
References
DS8000 Subsystem Storage Hierarchy
Storage Subsystem Hierarchy
Storage Complex
One or multiple physical storage subsystems
Central management point with Network Server
DS8000 Hardware Management Console (HMC)
S to ra g e
E n c lo su re
S to r a g e
E n c lo su r e
S to ra g e
E n c lo su re
S to r a g e
E n c lo su r e
S to ra g e
E n c lo su re
S to r a g e
E n c lo su r e
S to ra g e
E n c lo s u re
S to r a g e
E n c lo s u r e
CE C 0
CE C 0
CE C 1
CE C 1
I/O D r a w e r
R IO 1
I/O D r a w e r
R IO 1
I/O D ra w e r
R IO 1
I/O D r a w e r
R IO 0
I/O D r a w e r
R IO 0
I/O D ra w e r
R IO 0
I/O D ra w e r
R IO 1
I/O D ra w e r
R IO 0
Storage Subsystem Hierarchy (continued)

Storage Unit (Storage facility)
Single physical storage subsystem
Storage Image
Single logical storage subsystem
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Same as physical subsystem for 2107

921/931, 2107 922/932, 2107 941/94E
CEC 0
Server
CEC 1
Server0 and Server1

Manage extent pools and volumes
Even numbers managed by Server0

Odd numbers managed by Server1
I/O Drawer
RIO 1
I/O Drawer
RIO 1
I/O Drawer
RIO 0
I/O Drawer
RIO 0
DS8000 Array Site

Logical grouping of disks
Same capacity and speed
Created and assigned to DA pair by

software during installation
DS8000 array site
DS8000 array site

8 disks (DDMs)
FDE????
Array
DS8000 8 DDM arrays = 1 Array Site
RAID5
6+P
7+P
Parity is striped across all disks in array but consumes capacity
equivalent to one disk
RAID5 array performs better for sequential write
RAID 6
6+P+Q
5+P+Q+Spare
RAID10
3+3
4+4
RAID10 array performs better for random write
RAID5 and RAID10 arrays perform equally for read
Sparing
Arrays without spares may mean better potential random performance
Array capacity should be confirmed after array creation
Array types (with spares and without spares) should be balanced
across server0 and server1 (when ranks are assigned to extent pools)
Minimum of 4 spares per DA (64 or 128 disks) (single disk type)
D
D
D
D
D
P
D
S
RAID5 6+P+S
D
D
D
D
D
D
D
P
RAID5 7+P
D
D
D
D
D
D
P
Q
RAID6 6+P+Q
D
D
S
D
D
P
Q
RAID6 5+P+Q+S
D
D
D
D
D
D
S
S
RAID10 3+3+S+S
D
D
D
D
D
D
D
D
RAID10 4+4
8
Rank
RAID array with CKD or FB storage type defined
One-to-one relationship between an array and a rank
One RAID array becomes one rank (DS8000 8 DDMs)
A rank has no relationship to server0 or server1 until after it has been

assigned to an extent pool
Rank ID (Rx) does not indicate a server association unless specifically configured to do
so
Ranks should be assigned to server0 and server1 extent pools in a balanced manner
Ranks built on arrays containing spares should be balanced across server0 and server1 extent pools
DS8000 ranks built on array sites associated with each Device Adapter should be balanced across
server0 and server1 extent pools
DS8000 CKD rank

D
RAID5 7+P
A rank has no relationship to Logical Subsystems (LSSs)
Ranks are divided into extents
Units of space for volume creation
CKD rank
Extents equivalent to a 3390M1

1113 cylinders or .94GB
FB rank
1GB extents
Extent Pool
Logical grouping of extents from one or more ranks from
which volumes will be created
Ranks are assigned to extent pools
Pool contains one or more ranks
Extent Pool
Extent pool is one storage type

CKD or FB
LUN size is not limited to rank size

If more than one rank assigned to pool
Extents
User-assigned to Server0 or Server1

Extent Pool ID Px
If x is even, assigned to Server 0 and will support even-numbered LSSs
If x is odd, assigned to Server 1 and will support odd-numbered LSSs
Extent Pool can have a user nickname

Activity should be balanced across both servers
Minimum of 2 extent pools required to utilize system resources
Maximum number of ranks in a single pool should be the total
number of ranks to balance system resources
10
DS8000 Logical Subsystem/Logical Control Units

Similar to ESS:
LCU used for CKD and LSS for FB but are the same concept
LCU/LSS has a maximum of 256 addresses
LCU/LSS is the basis for Copy Services paths and consistency groups
For open systems, LSSs do not directly affect application performance
For System z, more LCUs will provide additional addresses for PAVs which can improve performance
Aliases (Parallel Access Volumes/PAVs) are shared within an LCU

Each device including an Alias has a Unit Control Block (UCB)
Purpose is similar to open I/O Queue depth
Logical Subsystem ID xy
x indicates the address group

An address group is a pre-determined set of 16 LCUs/LSSs (x0-xf) of the same storage type (all CKD or all FB) *
y indicates server assignment
If y is even, LSS is available with Server0 extent pools
If y is odd, LSS is available with Server1 extent pools
DS8000:
LCU/LSS does not have a pre-determined relationship to rank/DA pair
Up to 255 LCU/LSSs available
FB LSSs are automatically created during LUN creation
e.g. Creation of volume 1000 results in creation of LSS 10
CKD LSSs are explicitly defined
Allows specification of LCU/LSS type and SSID

* ESCON devices must be in Address Group 0 (LCU 00-0F)
11
Volume/LUN
Created from extents in one extent pool
Extent Pool
Rank
Volumes/LUNs can be larger than the size of a rank (if

multiple ranks are in one extent pool)
DS8000 introduced with CKD max size 64K cylinders or 56GB (with
appropriate software support)
DS8000 with R3.1 has CKD max size 262,668 cylinders or 223 GB
(with appropriate software support)
FB max size 2TB
Volumes/LUNs can be presented to host server in cylinder,

100MB or block granularity
Space is allocated in 1GB extents (FB) or 1113 cylinder extents (CKD)
2
A
1
0
Extent Pool
Rank
Rank
Volume ID
User specifies 4-digit hex volume ID which includes address group,
LCU/LSS and device ID:
xyzz
x=Address Group
xy=LCU/LSS
> Even LCU/LSSs are available for Server0 extent pools
> Odd LCU/LSSs are available for Server1 extent pools
zz=device ID
12
2A11
Logical Volume Considerations

Volume/LUN size
Volume size does not necessarily affect performance
For open systems, for a given amount of capacity, choose a volume size
small enough to allow volumes to be spread appropriately across all ranks
available to an application workload
For System z, larger volumes may require more aliases (PAVs)
Volume/LUN placement
Logical volume placement on ranks, DAs and servers (server0 and
server1) has an effect on performance
More drives per volume will improve performance
Logical volumes for each application workload should be allocated

according to isolation, resource sharing and spreading principles
13
Logical Configuration Considerations
14
Easy DS8000 Configuration

Extent Pool Planning Decisions

As few pools as possible
Pairs of pools (1/2 for DS8000 Server 0, for DS8000 Server 1 for balance)
Separate pairs of pools for:

z/OS (CountKeyData) and Open systems (Fixed Block)

Disk drive classes (type, size, speed)
RAID type (RAID5, RAID6, RAID10)
Standard (full provisioning) or Extent Space Efficient (thin provisioning) Storage Allocation Method
Rotate Extents or Rotate Volumes Extent Allocation Method
Rotate Extents recommended except for SVC, Oracle ASM, DB2 BCU and small, hot
volumes
Easy Tier
Volume Planning Decisions

Storage Allocation Method - Standard (full provisioning), Extent Space Efficient (thin provisioning) or Track
Space Efficient (FlashCopy targets)
Extent Allocation Method - Rotate Extents or Rotate Volumes (Rotate Extents recommended except for SVC,
Oracle ASM, DB2 Balanced Configuration Unit and small, hot volumes)
z/OS Logical Control Unit (LCU) and Open Systems Logical Subsystem ID - Copy Services Consistency
Groups
z/OS LCU and Subsystem ID z/OS system definitions
15
Easy DS8000 Configuration (2)
z/OS Storage
Open Systems Storage
1. Configure DS8000 I/O ports for z/OS

access
1. Configure DS8000 I/O ports for open

systems access
2. Create Extent Pools
2. Create Extent Pools
3. Create Logical Control Units (LCUs),

Count Key Data (CKD) Volumes
Parallel Access Volume (PAV) Aliases
3. Create DS8000 Host Definition and

DS8000 Volume Group (LUN masking)
16
4. Create Fixed Block Volumes
Principles of DS Performance Optimization

Allocation of logical volumes and host connections for an application
workload
Isolation
Spreading
These principles are described in detail in Chapter 4 (4.1-4.3) of

IBM TotalStorage DS8000 Series: Performance Monitoring and Tuning SG24-7146
17
Workload Isolation
Dedicating a subset of hardware resources to one workload
Ranks
I/O ports

Logical volumes/LUNs and host connections for the workload are

isolated to the dedicated resources
Provides increased probability of consistent response time for an

important workload, but
Maximum potential performance limited to the set of dedicated resources
Contention still possible for any resources which are not dedicated (e.g. processor,
cache, persistent memory)
18
Workload Isolation (2)

Can prevent less important workloads with high I/O demands from
impacting more important workloads
It may be acceptable for multiple less important, I/O intensive resources to contend
with each other on a single set of shared resources (isolated from other workloads)
A good approach if workload experience, analysis or modeling identifies:

A workload which tends to consume 100% of resources available
A workload which is much more important than other workloads
Conflicting I/O demands among workloads
DA level isolation may be appropriate for large blocksize, heavy

sequential workloads
19
Workload Spreading
Workload is balanced and distributed evenly across all
allowed hardware resources
Applies to both isolated and resource-sharing workloads
Logical volumes/LUNs
Logical volumes for a workload are spread across:
Ranks
Device adapters
Server0 and server1
Host logical volume striping should also be used
20
Workload Spreading (2)

Host connections
Host connections for a workload are spread across:
I/O ports
Host adapters
I/O enclosures
Server0 and Server1
Left side I/O enclosures and right side I/O enclosures
New host connections are allocated on least-used shared resources
For optimal performance:

Must use multiple paths
Configure ports on even I/O Bay and odd I/O Bay
Do not use all of the ports on a HA (bandwidth)

Do not mix PPRC links with host connections on the same HA to avoid contention
Use multipathing software
21
DS8000 Host Adapter

Port Assignment and HA Card Installation Sequence
DS8100
All host adapter cards are installed in base frame (Rack 1)
Optional expansion frame does not support additional host adapter cards
Maximum port count is 64
DS8300
Host adapter cards may be installed in Rack 1 or Rack 2 (if present)
DS8700
Rack 1 optionally may have only one pair of I/O enclosures (FC #1301 Qty 1)
DS8800
Rack 1 optionally may have only one pair of I/O enclosures (FC #1301 Qty 1)
22
1/12/2011
DS8000 Host Adapter

Port Assignment Best Practices
Isolate host connections from remote copy connections (MM, GM, zGM, GC
and MGM) on a host adapter basis
Isolate CKD host connections from FB host connections on a host adapter
basis
Always have symmetric pathing by connection type (i.e., use the same
number of paths on all host adapters used by each connection type). For
z/OS, all path groups should be symmetric (i.e., uniform number of ports
per HA) and spread path groups as widely as possible across all CKD host
adapters.
When possible, use the same number of host adapter connections
(especially for System z) as the number of connections coming from the
hosts.
Size the number of host adapters needed based on expected aggregate
maximum bandwidth and maximum IOPS (use Disk Magic or other common
sizing methods based on actual or expected workload.
Sharing different connection types within an I/O enclosure is encouraged.
When possible, isolate asynchronous from synchronous copy connections
on a host adapter basis.
When utilizing multipathing, try to zone ports from different I/O enclosures
to provide redundancy and balance (i.e., include port from a host adapter in
enclosure 0 and enclosure 1).
23
1/12/2011
DS8000 Host Adapter

Port Usage Best Practices
For optimal performance
Plug Order
Plug Order
(4 port card)
(8 port card)
Avoid using adjacent ports (if possible)
Port 1
For 8 port cards, use each 4 port set

equally
Port 2
Port 3
Port 4
Follow the following plug order sequence
24
Port 5
Port 6
Port 7
Port 8
1/12/2011
DS8000 Host Adapter

Port Usage Best Practices
Base Frame Only
Base and Expansion Frame
Port #
Port #
Path 1
I0230
I0230
Path 2
I0300
I0300
Path 3
I0030
I0030
Path 4
I0100
I0100
Path 5
I0200
I0630
Path 6
I0330
I0700
Path 7
I0000
I0430
Path 8
I0130
I0500
For System Z
Place each path in a path group in
a different I/O bay.
Do not have two paths from the
same path group sharing a card
This will balance the workload over
the available HBAs
25
1/12/2011
Defining Extent Pools on a DS8000
26
Extent Pool
Logical grouping of extents from one or more ranks from which volumes
will be created
Ranks are assigned to extent pools
Pool contains one or more ranks
Extent pool is one storage type
CKD or FB
LUN size is not limited to rank size
If more than one rank assigned to pool
Rotate Extents or Storage Pool Striping introduced with R3
Extent Pool
Rank
Preferred method of volume allocation and default with R6
Rotate Volumes method still available

Explicitly specify to insure you invoke the method you want
User-assigned to Server0 or Server1
Extent Pool ID Px
If x is even, assigned to Server 0 and will support even-numbered LSSs

If x is odd, assigned to Server 1 and will support odd-numbered LSSs
Extent Pool can have a user nickname

Activity should be balanced across both servers
Minimum of 2 extent pools required to utilize system resources
Maximum number of ranks in a single pool should be the total number of ranks to balance
system resources
27
DS8000 Rotate Volumes

Rank R0
Single volume is created from extents on one rank if

possible
2A10
2A11
In single-rank extent pool, multiple volumes will be

created sequentially on the rank
Rank R0
Single volume may spill across ranks in pool or

may be larger than the size of a single rank
In a multiple-rank extent pool, current
implementation places multiple volumes on rank
with most free extents
Volumes may dynamically deleted and extents
reused
2A10
2A11
Rank R2
Rank R0
2A10
2A11
Possible performance degradation due to hot

spots and limited resources per volume
28
Rank R2

4
DS8000 Storage Pool Striping

Preferred algorithm choice for volume

creation with R3
5
1
Rank 9
Naming
Marketing material Storage Pool Striping
DS CLI & DS Storage Manager Rotate Extents
Volumes are created by allocating one Extent

from available Ranks in an Extent Pool, in a
round-robin fashion
At right - 7 GB Volume showing the order of Extent

allocation
6
Rank 10
2
Rank 11
3
Extent Pool with 3 Ranks
CKD and Fixed Block
29
Storage Pool Striping - Characteristics

The next Volume will be started
from an Extent on the next Rank in
the round-robin rotation
2nd Volume start
If a Rank runs out of extents, it is

skipped
Multiple Volume allocations will
not start on the same Rank
If many Volumes are created with a
single command, the Volumes will not
start on the same Rank
Supports new Volume Expansion

capabilities
30
Extent pool with 3 Ranks
Storage Pool Striping - Advantages

Technical Advantages
Method to distribute I/O load across multiple Ranks
DS8000 optimized for performance
Far less special tuning required for high performance data placement.
Means less work for the storage administrator
Reduces storage administrator work needed to optimize performance
31
Storage Pool Striping Considerations & Recommendations (1)

Deleting Volumes creates free Extent units for
future volume allocation
DA2
DA2
reorg with Easy Tier manual mode

If ranks are added to extent pool reorg the
volumes using Easy Tier
DA0
DA0
DA3
DA3
ExtPool 0
ExtPool 1
Do not mix striped and non-striped volumes in

same multi-Rank Extent Pool
32
Storage Pool Striping Considerations & Recommendations (2)
Not recommended for:

SVC
When providing LUNs to the SVC, current best practice is to not use SPS and just give the SVC LUNs
that are an entire RAID5 array
Specific vendor Volume layout recommendations/requirements

DB2 BCU
Oracle ASM
Applications where OS level striping would be better choice

Small, hot Volumes
33
DS8700 Capacity Magic
34
DS8700 Capacity Magic
35
Spreadsheet
36
One Possibility
37
Or another.
38
Or Why Not?.
Large Extent Pools are easier to manage and make life easier
39
Mixed Capacity DDMs Equal Capacities
40
Capacity Report for Previous Config
41
Spreadsheet
42
Considerations/Observations
SATA isolated on DA 2
Could result in contention or overrun
Do not mix SATA and FC in the same Extent Pool

4 Extent Pools
Do not do frequent full box FlashCopy of volumes on FC to targets

on SATA
Better to use larger FC drives to spread BACKGROUND COPY over all ranks
43
44

Standard Logical Volumes
Track Space Efficient Logical Volumes
Used with Space-Efficient Flash Copy
Extent Space Efficient Logical Volumes

Used with Thin Provisioning
45
Standard Logical Volume (LV)

Standard LV consists of 1 to N Real Extents
Each Extent contains Extent Data and Extent
Metadata
Each LV Extent is mapped to a Real Extent

on a Real Rank
All Extents allocated to a LV come from one

Extent Pool
FB Extent
= 1024 MB Extent Data
Extent Pool
Standard LV
M
M
M
Real Rank
M
M
M
M
M
M
Less 4.5MB Extent Metadata
CKD Extent = 1113 Cylinders Extent Data
Extent Extent
Metadata
Data
Less 64 cylinders Extent Metadata

46
TSE and ESE

TSE
Storage allocated in track increments
Storage obtained from Repository
Repository uses extents obtained from extent pool
Intended to be used as FlashCopy targets
ESE
Storage allocated in extent increments
Storage obtained directly from extent pool
Currently supports FB only
Currently no copy services support for ESE volumes
47
Storage Resource Summary

Disk
Individual DDMs
Array Sites
Pre-determined grouping of DDMs of same speed
and capacity (8 DDMs for DS8000)
Arrays
RAID5, RAID6 or RAID10
One 8-DDM Array Site used to construct one RAID

array
Ranks
One Array forms one CKD or FB Rank
CKD or FB
No fixed, pre-determined relation to LSS
Extent Pools
1 or more ranks of a single storage type (CKD or
FB)
Assigned to Server0 or Server1
48
Extent
Pool
Storage Resource Summary (continued)

Volumes or LUNs
Made up of extents from one extent pool
Min allocation is one extent -- 1GB(FB) Mod1(CKD)
Max size is 2TB (FB); 223GB(CKD)
Can be larger than 1 rank if more than 1 rank in pool
Associated with LCU/LSS during configuration
Available LSSs determined by Extent Pool server affinity
Can be individually deleted
Open Systems Volume Group

Contains LUNs and host attachments -- FB LUN masking
One host attachment (one port or port group) can be member of
only one volume group
AIX host port
AIX host port
iSeries host
port group
One volume can be member of multiple volume groups
Multiple hosts can be contained in a single volume group

FB (i)
FB
FB
FB
FB
FB
49
Recommendations
Use the GUI for configuration
Create extent pools using storage pool striping with multiple ranks in the pool
Balance extent pools and ranks across servers
An extent pool for each server
Use a limited number of device types for ease of management
Use separate pools for DDMs of different size
Use large FC DDMs and large Extent Pools

Use custom volumes that are even multiple of extents
CKD 1113 cylinder extents
3390 M3
3390 M9
30051 cylinders
60102 cylinders
FB 1 GB extents
Use PAVs to allow concurrent access to base volumes for System z

Preferably HyperPAV
z/OS, z/VM and Linux for System z

50
Addendum
Logical Configuration of CKD Volumes
Using DS8000 Storage Manager
Logical Configuration of Open Systems Hosts and Volumes
Logical Configuration of CKD Extent Pools and Volumes
Using DS CLI
DS8000 Volumes with System z
References
51
Logical Configuration of CKD Volumes

52
Select Storage Complexes
53
Select DS8000 and Configure I/O ports
54
Select Port(s) and Change to Desired Protocol (FCP or FICON)
55
Create Extent Pools - z/OS
56
Select Disk Configuration
57
Select Create Extent Pools
58
CKD Extent Pool Creation
59
1.
Select CKD Storage Type
2.
Select Capacity to Configure
3.
Specify Pool Nickname_prefix

& Nickname Suffix
Leave other defaults
Verify and Select Create All
60
New Pools Would Appear Here (we cancelled out of creation)
61
Create Logical Control Units (LCUs) & Volumes and Aliases

(PAVs) System z
62
Select System z Volumes and LCUs
63
LCU Creation
1
64
1.
Select LCUs from Available LCU

Addresses (multiples of 2, one even,
one odd)
2.
Specify Subsystem ID (SSID)

according to your conventions
CKD Volume and Parallel Access Volume (PAV) Creation
1.
Select Rotate Extents Extent

Allocation Method
2.
Check box for Assign alias volumes

to these base volumes
3.
Assign alias ratio as desired (1/1 is a

good starting point)
4.
Specify volume Nickname prefix

and Nickname suffix
Leave other defaults or change if

desired
1
2
65
Verify and Continue
66
Leave Default Rotate LCUs in pools
67
68
Logical Configuration of Open Systems Hosts and Volumes

69
Select Disk Configuration
70
Select Create Extent Pools
71
Select Capacity to Configure & Specify Nickname prefix & Suffix

Take defaults for everything else
72
73
New Pools Would Appear Here (we cancelled out of creation)
74
Host Creation (Open Systems only)
75
Select Host Connections
76
Open Systems Host Connection
1
3
4
1. Specify Host Connection Nickname

2. Select Port Type Fibre Channel Point-to-Point
3. Select Host Type
4. Select Host WWPN and Add (repeat as necessary)
77
Select Map to a new volume group (LUN Masking)
78
Specify Volume Group Nickname for the new DS8000 Volume Group
79
Leave the Default Automatic (let zoning control host access) & Continue
80
Verify and Select Finish
81
Heres the Host
82
Volume Creation Open Systems
83
Select Open Systems Volumes & Create new volumes
84
Leave the default Rotate volumes in pools and Select the 2 Extent Pools
created earlier
85
Open Systems Volume Creation
4
1
1.
Specify Volume Size
2.
Specify Volume Quantity
3.
Select Rotate Extents
4.
Specify Volume Nickname_prefix & Nickname Suffix
5.
Select the Volume Group created earlier
86
Verify and Select Next
87
Take the default LSS selections (Even and Odd for Balance)
Due to other requirements, we needed to uncheck the 2 default

selections and select another LSS in each table (even and odd for
balance)
88
89
Select Manage Existing Volumes
90
Select Filter by Host Connection and Select Host Created Earlier to

see that the new volumes are mapped to the new host
91
Logical Configuration of CKD Extent Pools and Volumes

Using DS CLI
92
DS8000 DS CLI
Powerful tool for automating configuration tasks and collecting
configuration information
Same DS CLI for DS6000 and for ESS 800 Copy Services
DS CLI commands can be saved as scripts which significantly reduces the
time to create, edit and verify their content
Uses a consistent syntax with other IBM TotalStorage products now and in
the future
All of the function available to the GUI is also available via the DS CLI
93
Supported DS CLI Platforms

The DS Command-Line Interface (CLI) can be installed on the following
operating systems:
AIX 5.1, 5.2, 5.3
HP-UX 11i v1, v2
HP Tru64 version 5.1, 5.1A
Linux (RedHat 3.0 Advanced Server (AS) and Enterprise Server
(ES)
SUSE Linux SLES 8, SLES 9, SUSE 8, SUSE 9)
Novell Netware 6.5
Open VMS 7.3-1, 7.3-2
Sun Solaris 7, 8, 9
Windows 2000, Windows Datacenter, and Windows 2003
94
CKD Logical Configuration Steps

Creating CKD extent pools
Creating arrays
Creating and associating ranks with extent pools
Creating logical control units
Creating CKD volumes
Creating CKD volume groups (system generated)
95
Creating CKD extent pools

Remember from earlier?
Minimum of 2 extent pools required
Server0 extent pools will support even-numbered LSSs
Server1 extent pools will support odd-numbered LSSs
Consider creating additional extent pools for each of the following conditions:
Each RAID type (5, 6 or 10)
Each disk drive module (DDM) size
Each CKD volume type (3380, 3390)
Each logical control unit (LCU) address group
mkextpool -dev IBM.2107-75nnnnn rankgrp 0 -stgtype ckd P0

mkextpool -dev IBM.2107-75nnnnn rankgrp 1 -stgtype ckd P1
lsextpool dev IBM.2107-75nnnnnn -l
96
Creating Arrays
RAID array
RAID5, RAID6 or RAID10
Created from 1 array site on DS8000
Array Site
Logical grouping of disks
Same capacity and speed
Issue the lsarraysite command to find the unassigned array sites

lsarraysite -dev IBM.2107-75nnnnn -state unassigned
Issue the mkarray command to create an array

from each site with the status unassigned
mkarray -dev IBM.2107-75nnnnn -raidtype 5 -arsite A1
lsarray dev IBM.2107-75nnnnn l A1
97
Creating CKD Ranks

RAID array with storage type defined
One-to-one relationship between an array and a rank
One RAID array becomes one rank

DS8000 8 DDMs
Ranks have no pre-determined or fixed relation to:
CKD or FB
Server0, Server1 or Logical Subsystems (LSSs)
Ranks are divided into extents
Units of space for volume creation
CKD rank extents equivalent to a 3390M1
1113 cylinders or .94GB
Issue the lsarray command to find unassigned arrays

lsarray -dev IBM.2107-75nnnnn -state unassigned
Issue the mkrank command to assign a rank to rank group 0 or 1
according to the rank group number of the assigned extent pool ID.
mkrank -dev IBM.2107-75nnnnn -array a1 -stgtype ckd -extpool p1
lsrank -dev IBM.2107-75nnnnn -l
98
Creating CKD Logical Control Units (LCUs)

Up to 255 LCUs are available
LCU has a maximum of 256 addresses
Aliases (Parallel Access Volumes/PAVs) are shared within an LCU
Even-numbered LCUs are associated with Server0 and odd-numbered LCUs are associated
with Server1
LCU does not have a pre-determined relationship to rank/DA pair
Set of 16 Logical Subsystems (LSSs) is called an Address Group
LSS 00-0F, 10-1F, 20-2F, 30-3F, etc.

Storage type for entire Address Group (16 LSSs) is set to CKD or Fixed Block by the first LSS defined
CKD LSSs (LCUs) are explicitly defined
Allows specification of LCU type and SSID
Issue lsaddressgrp to find unassigned address groups

lsaddressgrp -dev IBM.2107-75nnnnn
99
Creating CKD LCUs (cont)

Analyze the report to identify all of the address groups that are available to be
defined. Use the following criteria:
If the list is empty, all of the address groups are available to be defined.
A defined address group with the storage type fb (fixed block) is not available to be defined.
A defined address group with the storage type ckd and with fewer than 16 LSSs is available for LCU
definition.
If you are using an undefined address group to make new LCUs, select the lowest numbered address group
that is not defined.
If you are defining a new LCU in an existing CKD address group, use the lslcu command to identify LCUs
that are already defined in the target address group.
Issue the mklcu command to create an LCU.

dscli>mklcu dev IBM.2107-75nnnnn -qty 16 -id 00 -ss 0010 -lcutype 3390-3
lslcu -dev IBM.2107-75nnnnn -l
100
Creating CKD Volumes

p1 Extent Pool
a1 Array
r1 Rank
00:0F LCUs
View your list of CKD extent pool IDs and determine which extent pool IDs that you
want to use as the source for the CKD volumes to be created. You obtained this list
when you first created your extent pools. If this list is not available, you can issue
the lsextpool command to obtain the list of extent pool IDs.
Issue the mkckdvol command to make 128 base and 128 alias volumes for each LCU.
Issue the mkckdvol command to create 128 3390 base volumes for the LCU.
mkckdvol -dev IBM.2107-75nnnnn -extpool p1 -cap 3339 -name finance#d 0100-017F
mkaliasvol -dev IBM.2107-75nnnnn base 0100-017F -order decrement -qty 2 01FF
lsrank -dev IBM.2107-75nnnnn -l
101
Initialize the CKD Volumes

Use ICKDSF to initialize the newly configured CKD Volumes
There is no VTOC, IXVTOC, VVDS or Volume Label at this time
To insure that you only initialize volumes without a label, specify
INIT UNITADDRESS(uadd) VOLID(volser) VFY(*NONE*) VTOC(n,n,nn) INDEX(n,n,nn)
102
Reference
IBM System Storage DS8700 Architecture and Implementation
SG24-8786
IBM System Storage DS8800 Architecture and Implementation
SG24-8886
103
DS8000 Volumes with z/OS
104
DS8000 Volumes with System z

System z supports both CKD and FB volumes (FB for zSeries Linux)
FB volumes are FCP attached, CKD volumes are FICON attached
Same 4GB 4 port FC Host Attachment Feature supports either FCP or FICON
Assigned at the Port level (FCP or FICON, single port cant be both)
FICON Fastloadnew method for Adapter Code Load
FICON FASTLOAD used for FICON attach

Compatible with FCP FastloadAllows intermix use of ports on HA
Architected event, no long busy used (yes)
Loss of Light less than 1.5 seconds for Adapter Code Load (only when Adapter Code is upgraded)
Concurrent Code Load Support

Advise all host attachments have (at least) two ports
Preferred on two separate Host Adapters
CKD also supported by ESCON Host Attachment Feature

105
DS8000 CKD Volumes

CKD standard volumes
3380
3390M3
3390M9
CKD custom volumes

Minimum volume size specification is 1 cylinder
Minimum space allocation is 1 extent (1113 cylinders)
Maximum volume size is 65,520 cylinders/56GB (DS8000 introduced with it)
With z/OS 1.4 or higher software support
Maximum volume size is 262,668 cylinders/223GB with R3.1
With z/OS 1.9+ or higher software support
Use a multiple of 1113 cylinders if possible
Maximum number of CKD volumes

64K per logical DS8000 *
* 4K limitation for for ESCON access

106
DS8000 z/OS HCD Considerations

New device support for D/T2107
Address Group 2 LCU 20 & 21

CNTLUNIT CUNUMBR=A000,PATH=(52,53,54,55),
DS8000 supports up to 16 Address Groups
64K logical volumes
For IOCP and HCD, the CU addresses are

hex 00 FE
LCU / LSS do not have to be contiguous
UNITADD=((00,256)),LINK=(24,34,25,35),
CUADD=20,UNIT=2107,DESC='N150 LCU20'
CNTLUNIT CUNUMBR=A100,PATH=(52,53,54,55),
UNITADD=((00,256)),LINK=(24,34,25,35),
CUADD=21,UNIT=2107,DESC='N150 LCU21
IODEVICE ADDRESS=((2000,128)),CUNUMBR=A000,
STADAT=Y,UNIT=3390B
STADAT=Y,UNIT=3390A
STADAT=Y,UNIT=3390B
STADAT=Y,UNIT=3390A
Examples provided at the DS8000 Information Center Search with IOCP

107
DS8000 z/OS HCD Considerations Subchannel Sets

Multiple Subchannel Sets
Z9 (2094) processor/ z/OS 1.7 only
HCD Implementation
Initial Implementation of SS1 req POR
Channel SubSystem (CSS) definition can contain Subchannel

Sets 0 & 1
256 Channels per CSS
Changed QPAVS display:

DS QPAVS,E278,VOLUME
IEE459I 09.57.53 DEVSERV QPAVS 046
HOST
SUBSYSTEM
CONFIGURATION
CONFIGURATION
--------------
--------------------
No changes to LSS definition in ESS, DS6000, DS8000
UNIT
UNIT
UA
Assign IODevice Base to Set 0
NUM. UA
TYPE
STATUS
SSID
ADDR.
TYPE
Assign IODevice Alias to Set 0 or 1
----- --
----
------
----
----
-----------
Duplicate Device Numbers Possible Desirable
0E278 78
BASE
3205
78
BASE
1E279 79
ALIAS-E278
3205
79
ALIAS-78
1E27A 7A
ALIAS-E278
3205
7A
ALIAS-78
0E27B 7B
ALIAS-E278
3205
7B
ALIAS-78
0E27C 7C
ALIAS-E278
3205
7C
ALIAS-78
0E27D 7D
ALIAS-E278
3205
7D
ALIAS-78
1E27E 7E
ALIAS-E278
3205
7E
ALIAS-78
ALIAS-E278
3205
7F
ALIAS-78
Relief for 64K Devices per LPAR
Providing they are in separate Subchannel Sets ...No

Problem
Flexible LSS structure
1E27F 7F
****
8 DEVICE(S) MET THE SELECTION CRITERIA
LPAR
Information provided in: z/OS V1R7.0 HCD Planning (GA22-7525-09)

0
108
Using larger volume sizes

Benefits
Fewer objects to define and manage
Less processing for fewer I/O resources
CF CHPID, VARY PATH, VARY DEVICE

Channel path recover, link recovery, reset event processing
CC3 processing
ENF Signals
RMF, SMF
Number of physical resources: CHPIDs, Switches, CU ports, fibers

Each device consumes real storage:
768 bytes of real storage for UCB and related control blocks
256 bytes of HSA
1024 bytes/device * 64K devices = 64MB
31 bit common storage constraints
EOV processing to switch to the next volume of a sequential data set significantly slows the access
methods
Considerations
Data migration to larger devices may be challenging, time consuming
109
zSeries Parallel Access Volumes (PAVs)

Additional addresses for a single device for improved performance
PAVs are shared within an LSS
An LSS may be on multiple ranks
Multiple LSSs may be on one rank
Recommendations
Use HyperPav if possible
z/OS, z/VM, Linux for System z
If not HyperPav use dynamic PAV for z/OS if possible
Requires parallel sysplex and WLM

Requires WLM having dynamic PAV specified
Requires WLM specified in device definition
110
DS8000 References
DS8000 Information Center
http://publib.boulder.ibm.com/infocenter/dsichelp/ds8000ic/index.jsp
Tutorials
Overviews
Publications and much more!
GC35-0515
DS8000 Introduction & Planning
GC26-7914
DS8000 Messages Reference
SC26-7917
DS8000 Host Systems Attachment Guide
SG24-6786
DS8000 Architecture & Implementation
SC26-7917
DS8000 Command-Line Interface Users Guide
DS8000 Code Bundle Information (Code Bundle, DS CLI, Storage Manager xref
http://www-01.ibm.com/support/docview.wss?uid=ssg1S1002949&rs=555
DS8000 Turbo Information (specs, white papers, etc.)
http://www-03.ibm.com/systems/storage/disk/ds8000/index.html
111
DS8000 References
Techdocs
http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs
PRS3574
IBM DS8000 + System z Synergy - March 2009
WP101528
IBM System z & DS8000 Technology Synergy
PRS3565 ATU - Storage Perf Mgmt with TPC
TD104162
Open System Storage Performance Evaluation
TD103689
Pulling TPC Performance Metrics for Archive and Analysis
Many more white papers, presentations and trifolds can be found on Techdocs!
112
SGA07 - Storage Subsystem Performance, Monitoring and Capacity

Planning for Open Systems
Learn both theoretical foundations in storage performance as well as
specific monitoring techniques using IBM TotalStorage Performance
Center (TPC). The course discusses essential performance
characteristics of cached disk subsystems, the essential performance
metrics, and enough theory to help understand why storage products
perform as they do. Moreover, the course covers the practical use of
TPC to monitor performance, to spot performance issues, and to
investigate the causes. Specific TPC reports and interpretation of the
reports are covered, as well as application of the data to long term
capacity planning. The students will have the opportunity to run
through a variety of hands on exercises with TPC as well.
http://www304.ibm.com/jct03001c/services/learning/ites.wss/us/en?pageType=course_search&sortBy=5&searchType=1&sortDirection=9&in
cludeNotScheduled=15&rowStart=0&rowsToReturn=20&maxSearchResults=200&language=en&country=us&searchString=sga07
113
Trademarks
The following terms are trademarks of International Business Machines Corporation in the United
States, other countries or both.
AS/400, DS6000, DS8000, DS Storage Manager, Enterprise Storage Server, FICON, FlashCopy,
GDPS, IBM, iSeries, pSeries, RS/6000, RMF, IBM TotalStorage, VM/ESA, VSE/ESA, xSeries, z/OS,
zSeries, z/VM, z/VSE On Demand Business
Intel and Pentium are trademarks of Intel Corporation in the United States, other countries, or both.
Microsoft, Windows and Windows NT are trademarks of Microsoft Corporation in the United States,
other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
114
Disclaimer
Copyright 2004 by International Business Machines Corporation.
No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation.
Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This
information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s)
and/or programs(s) at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or
withdrawal without notice, and represent goals and objectives only.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such such products,
programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this
document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does
not infringe IBM's intellectually property rights, may be used instead. It is the user's responsibility to evaluate and verify the operation of
any on-IBM product, program or service.
THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED.
IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR
NONINFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and
conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement,
etc.) under which they are provided. IBM is not responsible for the performance or interoperability of any non-IBM products discussed
herein.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or
copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.
115

DS8000 Logical Configuration 03152011

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DS8000 Logical Configuration 03152011

Uploaded by

Copyright:

Available Formats

Advanced Technical Skills (ATS) North America

DS8000: Storage Configuration and Best Practices

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

ATS Education - Customers

Tape Encryption with Tivoli Key Lifecycle Manager (TKLM)

TPC 4.1.1 Performance Management enhancements and

Ten Things for the new TPC Administrator to do to make TPC

Storage Subsystem Performance, Monitoring and Capacity

XIV Asynchronous Mirror

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

Logical Configuration Considerations

Defining Extent Pools on DS8000

Volume Virtualization Overview

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

DS8000 Subsystem Storage Hierarchy

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

Storage Subsystem Hierarchy

DS8000 Hardware Management Console (HMC)

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

Storage Subsystem Hierarchy (continued)

Same as physical subsystem for 2107

Server0 and Server1

Even numbers managed by Server0

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

DS8000 Array Site

Created and assigned to DA pair by

DS8000 array site

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

A rank has no relationship to server0 or server1 until after it has been

DS8000 CKD rank

Extents equivalent to a 3390M1

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

Extent pool is one storage type

LUN size is not limited to rank size

User-assigned to Server0 or Server1

Extent Pool can have a user nickname

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

DS8000 Logical Subsystem/Logical Control Units

Aliases (Parallel Access Volumes/PAVs) are shared within an LCU

x indicates the address group

e.g. Creation of volume 1000 results in creation of LSS 10

CKD LSSs are explicitly defined

Allows specification of LCU/LSS type and SSID

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

Volumes/LUNs can be larger than the size of a rank (if

Volumes/LUNs can be presented to host server in cylinder,

2010 IBM Corporation

Advanced Technical Skills (ATS) North America

Logical Volume Considerations

Logical volumes for each application workload should be allocated

2010 IBM Corporation

Advanced Technical Skills (ATS) North America