VIO Server IMT France

IBM GLOBAL SERVICES
IMT France / ITD La Gaude

Pseries Department
VIO server
Installation
Configuration
Standardization
Author : Alain Lentini
IBM France ITD La Gaude Unix team 2010
Author : Alain Lentini
Document Management
Changes history
Version
1.0
2.0
Date
May 2008
February 2010
author
A.Lentini
A.Lentini
Changes
Initial version DRAFT
- all types of SEA detailed with figures and build steps

- command line mode for all the SEA configuration
- rules and naming convention for SEA build
- SAN disks config detailed with 3 typical configuration
- command line mode for all the SAN disk configuration
- rules and naming convention for SAN build
- design and build recommendations for ITD/SSO lge
Reviewers list
Name
Ling Wai Hung
Stphane Saleur
Guy Aluno
Sylvain Desnoes
Nicolas Doz
Date
February 2010
February 2010
February 2010
February 2010
February 2010
-2-
1.
INTRODUCTION ..............................................................................................................7
2.
STANDARD DESIGN EXTRACTS ..................................................................................7
3.
INSTALL OS VIO .............................................................................................................9
4.
3.1.
DOWNLOAD FIXPACK AND VIOS ..................................................................................................... 9
3.2.
UPGRADE VIOS FIXPACK ................................................................................................................. 9
3.3.
VIOS BACKUP/RESTORE .................................................................................................................. 9
SHARED ETHERNET ADAPTER CONFIGURATION ....................................................9

4.1.
BASIC SEA WITHOUT VLAN TAGGING ............................................................................................ 9
4.1.1.
BASIC SEA LOGICAL VIEW .......................................................................................................... 9
4.1.2.
CREATE SEA ON VIO1 ................................................................................................................ 10
4.1.2.1.
Create virtual Ethernet adapter on VIO1 .......................................................................... 10
4.1.2.1.1. command line mode........................................................................................................ 11
4.1.2.2.
create control channel on VIO1 ........................................................................................ 11
4.1.2.2.2. rules and naming convention .......................................................................................... 12
4.1.2.3.
sea creation ........................................................................................................................ 13
4.1.3.
CREATE SEA ON VIO2 ................................................................................................................ 13
4.1.3.1.
4.1.3.2.
4.1.3.3.
sea creation ........................................................................................................................ 15
4.1.4.
CLIENT PARTITION CONFIGURATION ...................................................................................... 15
4.1.4.1.
4.1.4.2.
Create virtual Ethernet adapter on client partition .......................................................... 16

IP configuration on client partition ................................................................................... 16
4.2.
ETHERCHANNEL.............................................................................................................................. 16
4.3.
SEA WITH VLAN TAGGING ON THE SAME DEVICE ..................................................................... 16
4.3.1.
IMPORTANT : AVOID THIS SOLUTION FOR OUR ARCHITECTURES ..................................... 16
4.3.2.
SEA WITH VLAN TAGGING ON SAME INTERFACE LOGICAL VIEW ...................................... 16
4.3.3.
CREATE SEA ON VIO1 ................................................................................................................ 17
4.3.4.
CREATE ETHERCHANNEL ON VIO1.......................................................................................... 17
4.3.4.1.
4.3.4.2.
4.3.4.3.
sea creation on VIO1.......................................................................................................... 20
4.3.5.
CREATE SEA ON VIO2 ................................................................................................................ 20
-3-
4.3.6.
4.3.6.1.
4.3.6.2.
4.3.6.3.
4.3.7.
4.3.7.1.
4.3.7.2.
4.3.8.
4.4.

ADD A NEW VLAN ....................................................................................................................... 23

SEA WITH VLAN TAGGING RECOMMENDED SOLUTION ......................................................... 24
4.4.1.
SEA WITH VLAN TAGGING LOGICAL VIEW.............................................................................. 24
4.4.2.
WHY THIS SOLUTION IS RECOMMENDED................................................................................ 25
4.4.3.
CREATE SEA ON VIO1 ................................................................................................................ 25
4.4.4.
4.4.4.1.
Create virtual Ethernet adapters on VIO1 ........................................................................ 25
4.4.4.2.
4.4.4.3.
4.4.5.
CREATE SEA ON VIO2 ................................................................................................................ 28
4.4.6.
4.4.6.1.
4.4.6.2.
4.4.6.3.
4.4.7.
4.4.7.1.
4.4.7.2.
4.4.8.

ADD A NEW VLAN ....................................................................................................................... 32
4.4.8.1.
4.4.8.2.
Update SEA on VIO1 .......................................................................................................... 34
4.4.8.3.
4.4.8.4.
Update SEA on VIO2 .......................................................................................................... 35
4.4.8.5.
Client partition configuration ............................................................................................ 36
4.4.8.5.1. Create virtual Ethernet on client partition ........................................................................ 36
4.4.8.5.2. IP configuration on client partition ................................................................................... 36
5.
SAN DISKS CONFIGURATION.....................................................................................37

5.1.
RULES AND NAMING CONVENTION .............................................................................................. 37
-4-
5.1.1.
VTD NAMING CONVENTION ....................................................................................................... 37
5.1.2.
SLOT NUMBERS AND VSCSI SERVER...................................................................................... 37
5.1.2.1.
5.1.2.2.
5.2.
REDBOOK REFERENCES FOR RULES AND NAMING CONVENTION ......................................... 38
5.3.
VIRTUAL DEVICES ADAPTER VS QUEUE_DEPTH ....................................................................... 38
5.4.
IMPLEMENTATION IN TYPICAL CONFIGURATIONS ..................................................................... 39
5.4.1.
CONFIGURATION 1 ..................................................................................................................... 39
5.4.2.
CONFIGURATION 2 ..................................................................................................................... 40
5.4.3.
CONFIGURATION 3 ..................................................................................................................... 40
5.5.
FIBER CHANNEL AND HDISK DEVICES SETTINGS...................................................................... 41
5.6.
CONFIGURATION 2 DETAILED IMPLEMENTATION ...................................................................... 41
5.6.1.
CREATE VSCSI SERVER ADAPTER ON VIO1 FOR STORAGE B4 .......................................... 41
5.6.1.1.
5.6.2.
5.6.3.
command line mode .......................................................................................................... 47
CREATE CLIENT PARTITION DEVICE FOR VIO2 AND STORAGE G1..................................... 47
5.6.8.1.
5.6.9.
CREATE CLIENT PARTITION DEVICE FOR VIO1 AND STORAGE G1..................................... 46
5.6.7.1.
5.6.8.
CREATE VSCSI SERVER ADAPTER ON VIO2 FOR STORAGE G1.......................................... 45
5.6.6.1.
5.6.7.
CREATE VSCSI SERVER ADAPTER ON VIO1 FOR STORAGE G1.......................................... 44
5.6.5.1.
5.6.6.
CREATE CLIENT PARTITION DEVICE FOR VIO2 AND STORAGE B4 ..................................... 44
5.6.4.1.
5.6.5.
CREATE CLIENT PARTITION DEVICE FOR VIO1 AND STORAGE B4 ..................................... 43
5.6.3.1.
5.6.4.
CREATE VSCSI SERVER ADAPTER ON VIO2 FOR STORAGE B4 .......................................... 42
5.6.2.1.
CREATE VIRTUAL DEVICES ON VIO1 ....................................................................................... 47
5.6.10.
CREATE VIRTUAL DEVICES ON VIO2................................................................................... 48
5.6.11.
CLIENT PARTITION DEVICES SETTING................................................................................ 48
5.6.11.1.
5.6.11.2.
5.6.11.3.
5.6.11.4.
6.
rules with LPM.................................................................................................................... 37

Rules without lpm .............................................................................................................. 38
priority................................................................................................................................. 48
hcheck_interval .................................................................................................................. 49
queue depth........................................................................................................................ 49
vscsi_path_to ..................................................................................................................... 49
SCSI DISK DEVICES CONFIGURATION FOR ROOTVG CLIENT PARTITIONS .......49
-5-
6.1.
7.
CREATE THE LOGICAL VOLUMES AND THE VIRTUAL DEVICES............................................... 49
NTP CONFIGURATION ON VIO AND HMC .................................................................52

7.1.
NTP CONFIGURATION ON VIO SERVERS...................................................................................... 52
7.2.
NTP CONFIGURATION ON HMC...................................................................................................... 52
-6-
1.
Introduction
In this document, we assume that we must redundant VIO servers for each type of I/O.
This means we can have configuration with 2 VIO servers for LAN and 2 others VIO servers
for SAN, or we can have configuration with 2 VIO servers for LAN and SAN.
You can use the document to adapt to other types of configurations, but you manage the risks.
2.
Standard design extracts
Here are architecture designs recommended in this document.
Basic SEA without vlan tagging
-7-
SEA with vlan tagging recommended solution
Vscsi configuration 1 : 2 SAN VIO - 1 storage subsystem - 1 vscsi server per VIO server
-8-
Vscsi configuration 2 : 2 SAN VIO - 2 storage subsystems - 2 vscsi server per VIO
3.
install OS Vio
3.1.
download fixpack and vios
http://www14.software.ibm.com/webapp/set2/sas/f/vios/home.html
3.2.
upgrade vios fixpack
updateios -dev /mnt/lpp_source/vios14_fixpack92_lpp -install accept

updateios -dev /mnt/lpp_source/vios15_fixpack101_lpp -install accept
reboot
ioslevel
3.3.
VIOs BACKUP/RESTORE
backupios -file /mnt ( generate a tar file containing a mksysb and other informations )
The command creates a full backup tar file package including all of the resources that the installios
command will need to install a Virtual I/O Server (mksysb, bosinst.data, network bootimage, and
SPOT) from an HMC using the installios command. backupios file
/mnt/VIOS1_newP5_B2_220308.mksysb -mksysb ( generate only the mksysb file )
4.
Shared Ethernet Adapter configuration

4.1.
4.1.1.
Basic SEA without vlan tagging

Basic SEA logical view
-9-
4.1.2.
create SEA on VIO1
Initialize all physical lan interfaces with the good settings :

For example : 1000_Full_Duplex on the 2 VIOs
4.1.2.1.
Create virtual Ethernet adapter on VIO1
Create the Virtual LAN adapter on the Hmc ( with dlpar and in the Vio partition profile ) :
trunk priority = 1 on VIO1

A numerically lower priority receives a higher overall priority.
- 10 -
4.1.2.1.1.
command line mode
VIO Dynamic Device creation ( dlpar )

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 14 -a
"ieee_virtual_eth=1,port_vlan_id=1101,is_trunk=1,trunk_priority=1"
m=managed system
id=partition id
s=slot number
VIO Profile update
chsyscfg -r prof -m P01MMA03B49A04D07SN652F8BE -I
"name=Normal,lpar_id=2,virtual_eth_adapters+=14/1/1101//1/1"
virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required
4.1.2.2.
create control channel on VIO1
- 11 -
4.1.2.2.1.
command line mode

"ieee_virtual_eth=0,port_vlan_id=3101"
m=managed system
id=partition id
s=slot number
VIO Profile update
chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i
"name=Normal,lpar_id=2,virtual_eth_adapters+=
15/0/3101///1"
4.1.2.2.2.
rules and naming convention
slot number
all slots > 10
slots < 50 : LAN slots
slots > 50 : VSCSI slots
same slots on both VIO
PVID
x
1000 + Physical VLAN id
- 12 -
( Use virtual VLAN id different from the network VLAN on the switch )
This rule is mandatory for configuration with several SEA on the same VLAN.
For example, in case of VIO pro and pre-prod separated, you must use the same vlan on the
SEA of prod and pre-prod. So you must use different VLANid between prod and pre-prod
SEA.
PVID for control channel
x
4.1.2.3.
sea creation
This will generate a device on the VIO server ( cfgdev or cfgmgr )

To find the device, use the slot number in the command lsdev slots
Exemple lsdev -slots
U9117.570.65A1DA0-V2-C14 Virtual I/O Slot ent8 ( slot 2 )
U9117.570.65A1DA0-V2-C3 Virtual I/O Slot ent9
U9117.570.65A1DA0-V2-C15 Virtual I/O Slot ent13
lsdev -Cc adapter|grep ent
ent0 Available 0C-20 4-Port 10/100/1000 Base-TX PCI-X Adapter (14101103)
ent8
Available
Virtual I/O Ethernet Adapter (l-lan)
mkvdev -sea ent0 -vadapter ent8 -default ent8 -defaultid 1101 attr ha_mode=auto
ctl_chan=ent13
ent14
Important: If you do not define the ctl_chan attribute to the Shared Ethernet Adapters,
they will not be able to negotiate which one is providing the bridging functionality. At this
point, both SEAs will be bridging and a spanning tree loop can be formed. To avoid this,
always specify the these parameters with the SEA creation.
4.1.3.
create SEA on VIO2
Initialize all physical lan interfaces with the good settings :

For example : 1000_Full_Duplex on the 2 VIOs
4.1.3.1.
trunk priority = 2 on VIO2

A numerically lower priority receives a higher overall priority.
- 13 -
4.1.3.1.1.
command line mode

"ieee_virtual_eth=1,port_vlan_id=1101,is_trunk=1,trunk_priority=2"
m=managed system
id=partition id
s=slot number
VIO Profile update
chsyscfg -r prof -m P01MMA03B49A04D07SN652F8BE -I
"name=Normal,lpar_id=3,virtual_eth_adapters+=14/1/1101//2/1"
4.1.3.2.
- 14 -
4.1.3.2.1.
command line mode

m=managed system
id=partition id
s=slot number
VIO Profile update
"name=Normal,lpar_id=3,virtual_eth_adapters+=
15/0/3101///1"
4.1.3.3.
sea creation
mkvdev -sea ent0 -vadapter ent8 -default ent8 -defaultid 1101 attr ha_mode=auto
ctl_chan=ent13
ent14
Important: If you do not define the ctl_chan attribute to the Shared Ethernet Adapters,
they will not be able to negotiate which one is providing the bridging functionality. At this
point, both SEAs will be bridging and a spanning tree loop can be formed. To avoid this,
always specify the these parameters with the SEA creation.
4.1.4.
Client partition configuration
- 15 -
4.1.4.1.
Create virtual Ethernet adapter on client partition
4.1.4.2.
IP configuration on client partition
lscfg -vl ent*|grep ent

ent0 U9117.MMA.652F8CE-V27-C2-T1 Virtual I/O Ethernet Adapter
set IP address on the device ent0.
4.2.
etherchannel
mkvdev -lnagg ent8 ent12 ent16 -attr mode=8023ad
4.3.
4.3.1.
SEA with vlan tagging on the same device

important : avoid this solution for our architectures
This solution works properly but dont allow a dynamic update when you have to add a new
vlan on your network. It needs to delete the SEA to add a new vlan.
4.3.2.
SEA with vlan tagging on same interface logical view
- 16 -
4.3.3.
create sea on VIO1
4.3.4.
create etherchannel on vio1
Etherchannel VLAN 101/501

ent34
4.3.4.1.
Trunck priority 1
Not required
- 17 -
4.3.4.1.1.
command line mode

"ieee_virtual_eth=1,port_vlan_id=808,"addl_vlan_ids=101,501",is_trunk=1,trunk_priority=1"
VIO Profile update
"name=Normal,lpar_id=2,virtual_eth_adapters+=14/1/808/101,501/1/0"
4.3.4.2.
- 18 -

VIO Profile update
"name=Normal,lpar_id=2,virtual_eth_adapters+=16/0/895///1"
4.3.4.2.1.
The devices on VIO server must be set to not required.

slot number
all slots > 10
PVID
x

- 19 -
SEA.
decrement from 900
4.3.4.3.
sea creation on VIO1
lsdev slots
U9117.MMA.652F8CE-V2-C14 Virtual I/O Slot ent36
lsdev Cc adapter
ent34 Available
EtherChannel / IEEE 802.3ad Link Aggregation
mkvdev -sea ent34 -vadapter ent36 -default ent36 -defaultid 808 -attr ha_mode=auto
ctl_chan=ent38
ent39
4.3.5.
create sea on VIO2
4.3.6.
etherchannel VLAN 101/501

ent34
4.3.6.1.
Trunck priority 2
Not required
- 20 -
4.3.6.1.1.
command line mode

"ieee_virtual_eth=1,port_vlan_id=808,"addl_vlan_ids=101,501",is_trunk=1,trunk_priority=2"
VIO Profile update
"name=Normal,lpar_id=3,virtual_eth_adapters+=14/1/808/101,501/2/0"
4.3.6.2.
- 21 -

VIO Profile update
4.3.6.3.
lsdev slots
lsdev Cc adapter
ent34 Available
mkvdev -sea ent34 -vadapter ent36 -default ent36 -defaultid 808 -attr ha_mode=auto
ctl_chan=ent38
ent39
4.3.7.
4.3.7.1.

- 22 -
4.3.7.2.

4.3.8.
add a new vlan
As explained above, this action is not dynamic.

Thats why we recommend not to use this solution in our environments.
Modify the virtual device on VIO1 and VIO2. See example below :
- 23 -
Create the client virtual device and set the IP address on the client partition
4.4.
4.4.1.
SEA with vlan tagging recommended solution

SEA with vlan tagging logical view
- 24 -
4.4.2.
Why this solution is recommended
This solution is recommended because adding new vlan on the SEA is dynamic.
Even when a client network is stable, the client need can change at any time.
4.4.3.
create sea on VIO1
4.4.4.
Etherchannel VLAN 101/501

mkvdev -lnagg ent8 ent12 -attr mode=8023ad
ent34
4.4.4.1.
Create virtual Ethernet adapters on VIO1
Create one virtual Ethernet adapter per vlan on each VIO server.
Trunck priority 1
- 25 -
4.4.4.1.1.
command line mode

- 26 -
chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 14 a

"ieee_virtual_eth=1,port_vlan_id=1101,"addl_vlan_ids=101",is_trunk=1,trunk_priority=1"
VIO Profile update
"name=Normal,lpar_id=2,virtual_eth_adapters+=14/1/1101/101/1/1"
4.4.4.2.
4.4.4.2.1.
command line mode

VIO Profile update
- 27 -
4.4.4.2.2.
slot number
all slots > 10
PVID
x

SEA.
x
decrement from 900
4.4.4.3.
lsdev slots
lsdev Cc adapter
ent34 Available
mkvdev -sea ent34 -vadapter ent36 ent37 -default ent36 -defaultid 808 -attr ha_mode=auto
ctl_chan=ent38
ent39 ( new device genereted )
4.4.5.
create sea on VIO2
4.4.6.
etherchannel VLAN 101/501

mkvdev -lnagg ent8 ent12 -attr mode=8023ad
ent34
4.4.6.1.
Trunck priority 2
- 28 -
4.4.6.1.1.
command line mode
- 29 -

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 14 a
VIO Profile update
4.4.6.2.
4.4.6.2.1.
command line mode

VIO Profile update
- 30 -
4.4.6.3.
mkvdev -sea ent34 -vadapter ent36 ent37 -default ent36 -defaultid 808 -attr ha_mode=auto
ctl_chan=ent38
ent39 ( new device genereted )
4.4.7.
4.4.7.1.

- 31 -
4.4.7.2.
lsdev Cc adapter
4.4.8.
4.4.8.1.
add a new vlan

- 32 -
4.4.8.1.1.
command line mode

- 33 -

VIO Profile update
"name=Normal,lpar_id=2,virtual_eth_adapters+=17/1/1105/105/1/1
4.4.8.2.
Update SEA on VIO1
lsdev -slots
chdev -dev ent39 -attr virt_adapters=ent36,ent37, ent56, ent57
4.4.8.3.
- 34 -
4.4.8.3.1.
command line mode

VIO Profile update
"name=Normal,lpar_id=2,virtual_eth_adapters+=17/1/1105/105/2/1
4.4.8.4.
Update SEA on VIO2
lsdev -slots
chdev -dev ent39 -attr virt_adapters=ent36,ent37, ent56, ent57
- 35 -
4.4.8.5.
4.4.8.5.1.
Create virtual Ethernet on client partition
4.4.8.5.2.
- 36 -

ent1
U9117.MMA.652F8BE-V34-C5-T1 Virtual I/O Ethernet Adapter (l-lan)
ent0
U9117.MMA.652F8BE-V34-C4-T1 Virtual I/O Ethernet Adapter (l-lan)
set IP address on the device ent0 and ent1
5.
san disks configuration

5.1.
5.1.1.
vtd naming convention
Depending on your ios version and fixpack level, its possible that the vtd name can be
renamed by LPM. In this case, its useless to define vtd naming convention.
Depending on the customer specific context, it can be usefull that the vtd contains several
information such as the geographical or functional information.
Here is the recommended vtd syntax :
[ additional information ] [ _ ] < client partition name > [ _ ] < lun_id >
additional information : any geographical or functional information
client partition name : this name can be reduced if necessary
lun_id
: lun id on the storage subsystem
Examples :
usr210_0543
us : counry
r210 : a part of the client partition name
0543 : lunid on DS8000
saxsap12phd4
saxsap12p : a part of the client partition name
hd4 : corresponding disk on the client partition
5.1.2.
5.1.2.1.
slot numbers and vscsi server

rules with LPM
Vscsi server adapter slot number

No specific rule for slot number on VIO server if LPM is used (LPM redefines slot numbers).
Vscsi client adapter
- 37 -
x
x
Use Odd numbered slots starting at 21 for connection to the first VIO server, and use
even numbered slots starting at 20 for connection to the second VIO server.
Allocate slots in pairs, with each client using two adjacent slots such as 21 and 20, or
31 and 30.
5.1.2.2.
Rules without lpm
Vscsi server adapter

x
x
Use a specific VSCSI server for each storage subsystem.

The first VIOS should use Odd numbered slots number starting at 51, and the second
should use even numbered slots starting at 50. In a two server scenario, allocate slots
in pairs, with each client using two adjacent slots such as 51 and 50, or 61 and 60.
Vscsi client adapter

x
Use the same slot number between the vscsi server and client adapter.
5.2.
redbook references for rules and naming convention
see explain in the following redbook :

IBM System p Advanced POWER Virtualization Best Practices redp4194.pdf
chapter 4.3.2 : Virtual device slot numbers
After establishing the naming conventions, also establish slot numbering conventions for the
virtual I/O adapters.
Slot numbers are shared between virtual storage and virtual network devices. In complex
systems, there will tend to be far more storage devices than network devices because each
virtual SCSI device can only communicate with one server or client. We recommend
reserving the slot numbers through 20 for network devices on all LPARs in order to keep the
network and storage devices grouped together.
Management can be simplified by keeping slot numbers consistent between the virtual I/O
client and server. However, when partitions are moved from one server to another, this might
not be possible. For more information, see 4.6.3, Storage planning on page 108.
In environments with only one virtual I/O server, add storage adapters incrementally starting
with slot 21 and higher. When clients are attached to two Virtual I/O Servers, the adapter slot
numbers should be alternated from one VIOS to the other. The first VIOS should use odd
numbered slots starting at 21, and the second should use even numbered slots starting at 22.
In a two server scenario, allocate slots in pairs, with each client using two adjacent slots such
as 21 and 22, or 33 and 34.
5.3.
Virtual devices adapter vs queue_depth
As explained below, you have to take care of the maximum disks per virtual adapter server on
the VIO.
Use the following process to evaluate this max value : ( 512 2 ) / ( 3 + queue depth )
See redbook power VM Virtualization Managing and monitoring
sg247590.pdf chapter 2.5.2 Virtual device slot numbers:
- 38 -
Because virtual SCSI connections operate at memory speed, there is generally no

performance gain from adding multiple adapters between a Virtual I/O Server and client. For
AIX virtual I/O client partitions, each adapter pair can handle up to 85 virtual devices with
the default queue depth of three. For IBM i clients, up to 16 virtual disk and 16 optical
devices are supported. In situations where virtual devices per partition are expected to exceed
that number, or where the queue depth on some devices might be increased above the default,
reserve additional adapter slots for the Virtual I/O Server and the virtual I/O client partition.
When tuning queue depths, the VSCSI adapters have a fixed queue depth. There are 512
command elements of which 2 are used by the adapter, 3 are reserved for each VSCSI LUN
for error recovery and the rest are used for I/O requests. Thus, with the default queue depth
of 3 for VSCSI LUNs, that allows for up to 85 LUNs to use an adapter: (512 - 2) / (3 + 3) = 85
rounding down. So if you need higher queue depths for the devices, then the number of LUNs
per adapter is reduced. For example, if you want to use a queue depth of 25, that allows
510/28= 18 LUNs per adapter for an AIX client partition.
5.4.
5.4.1.
x
x
x
x
Implementation in Typical configurations

Configuration 1
2 SAN VIO
1 storage subsystem
1 vscsi server per VIO server
LPM used
vscsi1 U9119.FHA.832EE32-V234-C21-T1
vscsi0 U9119.FHA.832EE32-V234-C20-T1
Client adapter slot numbers
VIO2
VIO1
- 39 -
Storage B4
Storage B4
5.4.2.
x
x
x
x
configuration 2
2 SAN VIO
2 storage subsystems
2 vscsi server per VIO : 1 for each storage subsystem
LPM used
Storage bay B4 ( slot nb range 20 to 29 )

Storage bay G1 ( slot nb range 30 to 39 )
vscsi3 U9119.FHA.832EE32-V234-C31-T1
vscsi2 U9119.FHA.832EE32-V234-C30-T1
vscsi1 U9119.FHA.832EE32-V234-C21-T1
vscsi0 U9119.FHA.832EE32-V234-C20-T1
5.4.3.
x
x
x
x
VIO2
VIO1
VIO2
VIO1
Storage G1
Storage G1
Storage B4
Storage B4
configuration 3
2 SAN VIO
2 storage subsystems
4 vscsi server per VIO : 2 for each storage subsystems ( for nb device limitation )
LPM used
Storage bay B4 ( slot nb range 20 to 29 )

Storage bay G1 ( slot nb range 30 to 39 )
vscsi7
vscsi6
U9119.FHA.834EE9F-V132-C33-T1
U9119.FHA.834EE9F-V132-C32-T1
- 40 -
VIO2
VIO1
Storage G1
Storage G1
vscsi5 U9119.FHA.834EE9F-V132-C23-T1
5.5.
VIO2
VIO1
VIO2
VIO1
VIO2
VIO1
Storage B4
Storage B4
Storage G1
Storage G1
Storage B4
Storage B4
Fiber channel and hdisk devices settings
For each disk device on the vio server :

Activate the pvid on all the disks devices on both vioservers : chdev -l hdiskX -a pv=yes
Set the reserve_policy parameter : chdev -dev hdiskX -attr reserve_policy=no_reserve
For each Fiber channel card on the VIO server
Initialize all physical fiber channel interfaces with the good settings
chdev -dev fscsi0 -attr fc_err_recov=fast_fail dyntrk=yes perm
5.6.
Configuration 2 detailed implementation
Vio servers :
Client partition :
building :
dual storage :
LPM used
VIE1P01MMA04B4 VIE2P01MMA04B4
PODCLIENT_TEST6
B4 and G1
1 storage subsystem in each building
Doesnt need the same slot number between server and client adapter
In this example, the values of vscsi server adapters slot number is not important as LPM is
used.
5.6.1.
create vscsi server adapter on VIO1 for storage B4
The adapter must be not required for LPM
- 41 -
5.6.1.1.
command line mode
vio1 device, server slot 50, client slot 20, storage B4

chhwres -r virtualio -m 9119-FHA-SN834EE9F-BA1A05D12 -o a --id 4 --rsubtype scsi -s 50
-a "adapter_type=server,remote_lpar_name=PODCLIENT_TEST6,remote_slot_num=20"
chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i
"name=default,lpar_id=4,virtual_scsi_adapters+=50/server//PODCLIENT_TEST6/20/0"
5.6.2.
create vscsi server adapter on VIO2 for storage B4
- 42 -
5.6.2.1.
command line mode
vio2 device, server slot 51, client slot 20, storage B4

5.6.3.
create client partition device for VIO1 and storage B4
- 43 -
5.6.3.1.
command line mode
client partition device, vio1, server slot 50, client slot 20, storage B4
"name=default,lpar_id=128,virtual_scsi_adapters+=20/client//VIE1P01MMA04B4/50/1"
5.6.4.
5.6.4.1.
create client partition device for VIO2 and storage B4
command line mode
client partition device,vio2, server slot 51, client slot 21, storage B4
5.6.5.
create vscsi server adapter on VIO1 for storage G1
- 44 -
5.6.5.1.
command line mode
vio1 device, server slot 60, client slot 30, storage G1

5.6.6.
create vscsi server adapter on VIO2 for storage G1
- 45 -
5.6.6.1.
command line mode
vio2 device, server slot 61, client slot 31, storage G1

5.6.7.
create client partition device for VIO1 and storage G1
- 46 -
5.6.7.1.
command line mode
client partition device, vio1, server slot 60, client slot 30, storage G1
5.6.8.
5.6.8.1.
create client partition device for VIO2 and storage G1
command line mode
client partition device, vio2, server slot 61, client slot 31, storage G1
5.6.9.
create virtual devices on VIO1
lsdev -slots
U9119.FHA.834EE9F-V4-C50 Virtual I/O Slot vhost0
x
x
vhost0 will be used for B4 storage subsystem.

vhost5 will be used for G1 storage subsystem.
rootvg
hdisk13 B4
hdisk52 G1
su - padmin -c ioscli mkvdev -vdev hdisk13 -vadapter vhost0 -dev p10_0230 ;
ssh PODCLIENT_TEST6 cfgmgr
su - padmin -c ioscli mkvdev -vdev hdisk52 -vadapter vhost5 -dev p10_0231 ;
- 47 -

hdisk2 to hdisk6 : storage subsystem B4
su - padmin -c ioscli mkvdev -vdev hdisk2 -vadapter vhost0 -dev usp010_3102 ;
hdisk39 to hdisk43 : storage subsystem G1
5.6.10.
create virtual devices on VIO2
lsdev -slots
The vhosts are the same as the vhosts of VIO1, the script to generate the virtual devices is the
same.
5.6.11.
5.6.11.1.
Client partition devices setting

priority
Change the priority on all the disks :

You have to share the workload between both vioservers.
Here are the different strategies :
x Set all the disks of each partition on one vioserver with priority 1
x Spread all disks of each partition between both VIO servers
chpath -l hdisk0 -p vscsi5 -a priority=2
chpath -l hdisk1 -p vscsi5 -a priority=2
- 48 -
Check : lspath -AHE -l hdiskx -p vscsiY
5.6.11.2.
hcheck_interval
Change the following setting on each disk device :

hcheck_interval needs to be greater than the R/W timeout interval.
1) check the rw_timeout value on the physical device disks on the vio
2) put at least the same value on the hcheck_interval on the virtual device disks on the partition
chdev -l hdiskX -a hcheck_interval=60 P

Check
lsattr -El hdiskX| grep hcheck_interval
5.6.11.3.
queue depth
For all the device disks of the client partitions, set the queue depth at the same value as the
queue depth value of the physical disks of the VIO servers.
See recommendation redbook "Virt Best Practices Redp4194" chapter 4.8 SCSI queue depth:
We recommend that you have the same queue depth on the virtual disk as the physical disk
5.6.11.4.
vscsi_path_to
see redbook sg247940 advanced power virtualization chapter 4.2.1 MPIO in the client partition
On all the virtual scsi devices of the client partition, put the following value :
chdev -l vscsiX -a vscsi_path_to=30 P
6.
scsi disk devices configuration for rootvg client partitions
Create the vhost in WebSm menu.

properties -> Virtual IO Adapter->onglet scsi->create server adapter
Naming convention.You can use the following rules :
Start the slots numbers to 20 for the Vscsi devices. Use < 20 for the LAN devices.
Use even slots numbers for VIO1 and odd numbers for VIO2
Use the first slot number for Aix disk and the other for the swap disk
Set the maximum slot number to a value that prevents the VIO partition stop.
assign the new virtual device to its specific client partition.

On the VIOs : cfgdev
Check with lsdev -slots
6.1.
Create the logical volumes and the virtual devices
On vioserver1
- 49 -
Create the virtual server device in WebSm.

assign the new virtual device to its specific client partition. See example below :
In our naming convention, we advice to chose the same slot number for the client and for the
server virtual scsi adapter ( see example below ) :
mkvg -f -vg vioserver1VG hdisk1 hdisk5

extendvg vioserver1VG hdisk2 hdisk6
partition mopesaxsap15p
mklv -lv saxsap15phd0LV vioserver1VG 30G hdisk1
mkvdev -vdev saxsap15phd0LV -vadapter vhost0 -dev vtsaxsap15phd0
on vioserver2
Create the virtual server device in WebSm
mkvg -f -vg vioserver2VG hdisk1 hdisk5
extendvg vioserver2VG hdisk2 hdisk6
partition mopesaxsap15p
On the client partition mopesaxsap15p
Create the vscsi client on the client partition profile ( same slot number as the vscsi server )
- 50 -
Here is an example of the result after the creation of 4 virtual scsi client adapter :
- 51 -
7.
NTP configuration on VIO and HMC
Enable NTP on all VIO servers and all HMC.
7.1.
Ntp configuration on VIO servers
See redbook sg247590 power VM managing and monitoring

chapter 4.3 Network Time Protocol configuration
vi /home/padmin/config/ntp.conf
startnetsvc xntpd ( stopnetsvc xntpd )
check your ntp.log file
lsnetsvc xntpd
ntpdate d <server>
7.2.
NTP configuration on HMC
See redbook sg247491 HMC V7 handbook

chapter 2.5.14 Change Date and Time
- 52 -
- 53 -

VIO Server IMT France

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

VIO Server IMT France

Uploaded by

Copyright:

Available Formats

IBM GLOBAL SERVICES

IMT France / ITD La Gaude

IBM France ITD La Gaude Unix team 2010

Author : Alain Lentini

- all types of SEA detailed with figures and build steps

IBM France ITD La Gaude Unix team 2010

STANDARD DESIGN EXTRACTS ..................................................................................7

INSTALL OS VIO .............................................................................................................9

DOWNLOAD FIXPACK AND VIOS ..................................................................................................... 9

UPGRADE VIOS FIXPACK ................................................................................................................. 9

VIOS BACKUP/RESTORE .................................................................................................................. 9

SHARED ETHERNET ADAPTER CONFIGURATION ....................................................9

BASIC SEA WITHOUT VLAN TAGGING ............................................................................................ 9

BASIC SEA LOGICAL VIEW .......................................................................................................... 9

CREATE SEA ON VIO1 ................................................................................................................ 10

CREATE SEA ON VIO2 ................................................................................................................ 13

CLIENT PARTITION CONFIGURATION ...................................................................................... 15

Create virtual Ethernet adapter on client partition .......................................................... 16

SEA WITH VLAN TAGGING ON THE SAME DEVICE ..................................................................... 16

IMPORTANT : AVOID THIS SOLUTION FOR OUR ARCHITECTURES ..................................... 16

SEA WITH VLAN TAGGING ON SAME INTERFACE LOGICAL VIEW ...................................... 16

CREATE SEA ON VIO1 ................................................................................................................ 17

CREATE ETHERCHANNEL ON VIO1.......................................................................................... 17

CREATE SEA ON VIO2 ................................................................................................................ 20

IBM France ITD La Gaude Unix team 2010

CREATE ETHERCHANNEL ON VIO2.......................................................................................... 20

CLIENT PARTITION CONFIGURATION ...................................................................................... 22

Create virtual Ethernet adapter on client partition .......................................................... 22

ADD A NEW VLAN ....................................................................................................................... 23

SEA WITH VLAN TAGGING LOGICAL VIEW.............................................................................. 24

WHY THIS SOLUTION IS RECOMMENDED................................................................................ 25

CREATE SEA ON VIO1 ................................................................................................................ 25

CREATE ETHERCHANNEL ON VIO1.......................................................................................... 25

CREATE SEA ON VIO2 ................................................................................................................ 28

CREATE ETHERCHANNEL ON VIO2.......................................................................................... 28

CLIENT PARTITION CONFIGURATION ...................................................................................... 31

Create virtual Ethernet adapter on client partition .......................................................... 31

ADD A NEW VLAN ....................................................................................................................... 32

SAN DISKS CONFIGURATION.....................................................................................37

RULES AND NAMING CONVENTION .............................................................................................. 37

IBM France ITD La Gaude Unix team 2010

VTD NAMING CONVENTION ....................................................................................................... 37

SLOT NUMBERS AND VSCSI SERVER...................................................................................... 37

REDBOOK REFERENCES FOR RULES AND NAMING CONVENTION ......................................... 38

VIRTUAL DEVICES ADAPTER VS QUEUE_DEPTH ....................................................................... 38

IMPLEMENTATION IN TYPICAL CONFIGURATIONS ..................................................................... 39

FIBER CHANNEL AND HDISK DEVICES SETTINGS...................................................................... 41

CONFIGURATION 2 DETAILED IMPLEMENTATION ...................................................................... 41

CREATE VSCSI SERVER ADAPTER ON VIO1 FOR STORAGE B4 .......................................... 41

command line mode .......................................................................................................... 47

CREATE CLIENT PARTITION DEVICE FOR VIO2 AND STORAGE G1..................................... 47

command line mode .......................................................................................................... 46

CREATE CLIENT PARTITION DEVICE FOR VIO1 AND STORAGE G1..................................... 46

command line mode .......................................................................................................... 45

CREATE VSCSI SERVER ADAPTER ON VIO2 FOR STORAGE G1.......................................... 45

command line mode .......................................................................................................... 44

CREATE VSCSI SERVER ADAPTER ON VIO1 FOR STORAGE G1.......................................... 44

command line mode .......................................................................................................... 44

CREATE CLIENT PARTITION DEVICE FOR VIO2 AND STORAGE B4 ..................................... 44

command line mode .......................................................................................................... 43

CREATE CLIENT PARTITION DEVICE FOR VIO1 AND STORAGE B4 ..................................... 43

command line mode .......................................................................................................... 42

CREATE VSCSI SERVER ADAPTER ON VIO2 FOR STORAGE B4 .......................................... 42

command line mode .......................................................................................................... 47