Brocade 4Gb SAN

Switch for HP c-Class

BladeSystem:
Service and Support
Training
eLMS 31680

June 2006 Walton Little, Jr. and Steve Pomfret

HP Restricted
© 2006 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice
Introduction
Course content
• Product Overview – c-Class BladeSystem
• Product Overview – 4Gb SAN switch for c-Class
BladeSystem
• Installing the 4Gb SAN switch
• Configuring the 4Gb SAN switch
• Managing the 4Gb SAN switch

Rev. 6.21 HP Restricted 2

Objectives
Service Engineer will be able to
• Describe the c-Class BladeSystem and its components
• Describe features and physical configuration of the 4Gb
SAN switch
• Install and configure the SAN switch
• Understand the need to backup switch configuration
• Understand switch monitoring and management capabilities
and where to find procedures to execute those capabilities

Rev. 6.21 HP Restricted 3

Product overview –
c-Class
BladeSystem

Rev. 6.21 HP Restricted 4

c-Class BladeSystem
• Next generation follow-on to HP p-Class BladeSystem
• c-Class features
– Provides support for next generation blade servers to 2012
and beyond
– Provides automated and virtualized infrastructure
– Provides highly scalable data processing/storage options
– Supports a variety of blades
– Supports multiple redundant fabrics
– Supports multiple interconnect devices
– Options for single and three-phase power

Rev. 6.21 HP Restricted 5

c7000 BladeSystem enclosure (1 of 4)
• 10U enclosure
• Hosts
– Hot-pluggable power supplies (up to 6 in single-phase
configuration)
– Hot-pluggable fans (up to 10)
– Onboard Administrator modules
– Front LCD panel (also called the Insight Display)
– Storage blades (SAS and SATA drives supported)
– Server blades (SAS and SATA drives supported)
– Multiple fabric switches and pass-through modules

Rev. 6.21 HP Restricted 6

c7000 BladeSystem enclosure (2 of 4)
• Supports
– Half-height server blades
– Full-height server blades
– Half-height storage blades
• Bay numbering depends on
device size
– Full-height numbering – top
right illustration
– Half-height numbering –
bottom right illustration

Rev. 6.21 HP Restricted 7

c7000 BladeSystem enclosure (3 of 4)
Front components Full-height
server blades

• Up to 8 full-height server Half-height

server blades
Half-height
storage blades
blades
• Up to 16 half-height server
blades
• Up to 15 half-height storage
blades
– Up to 90 drives per
enclosure
• Mix of above components
• Up to 6 power supplies
• Front LCD panel Hot-pluggable Front LCD
power supply panel
– Used for system (6)

Rev. 6.21 management HP Restricted 8

c7000 BladeSystem enclosure (4 of 4)
8 interconnect
Rear components include Fan bays for switches
and pass-throughs
• Hot-pluggable fans (up to
10)
• Power connectors (up to 6
in single-phase
configuration)
• Redundant Onboard
Administrator modules
• Interconnect bays (8) for
switches and pass-through
modules Redundant Enclosure Redundant
Onboard links 3+3 C19 cords
Administrator or
modules 30A 3-Phase

Rev. 6.21 HP Restricted 9

Rear component bay numbering (1 of 4)
• Interconnect bay numbering

Interconnect bay Server blade connection

NIC 1 and NIC 3

1
(embedded)
NIC 2 and NIC 4
2
(embedded)

3 and 4 Mezzanine 1

5 and 6 Mezzanine 2

Mezzanine 3 (full-height
7 and 8
server blade only)

Rev. 6.21 HP Restricted 10

Rear component bay numbering (2 of 4)
• Fan bay numbering is shown below

Rev. 6.21 HP Restricted 11

Rear component bay numbering (3 of 4)
• Onboard Administrator bay numbering is shown below and
described in the table

Item Description

1 Onboard Administrator bay 1

2 Enclosure link-down port

3 Enclosure link-up port

4 Onboard Administrator bay 2

Rev. 6.21 HP Restricted 12

Rear component bay numbering (4 of 4)
• Single-phase power
configuration and
connector
numbering are
shown (top right)

• Three-phase power
configuration is
shown (bottom right)

Rev. 6.21 HP Restricted 13

Enclosure power
• Supports single-phase and three-phase configurations
• Uses enclosure power scheme versus separate rack-
provided power
– Reduces power requirement
• Rack-based power for the c-Class enclosure would require
60A-100A three-phase power
• Hot-pluggable power supplies provide power to
– Enclosure
– Server Blades
– Switches and pass-throughs
– Onboard Administrator modules
– Fans

Rev. 6.21 HP Restricted 14

Power supply
• 2250W output
• Self-cooled Power Backplane
• Self-monitoring connector

– AC input detection
AC Input Module
– DC output measurement connector

Rev. 6.21 HP Restricted 15

Power configurations
• Less than 6 Power Supplies is possible if less power is
needed
• When using less than 6 PSs, install units as described
– 2 power supplies: bays 1 and 4
– 3 power supplies: bays 1, 2, and 4
– 4 power supplies: bays 1, 2, 4, and 5
– 5 power supplies: bays 1, 2, 3, 4, and 5
• The front LCD panel slides left or right to allow access to all
power supply bays

Rev. 6.21 HP Restricted 16

Power redundancy
• Power configured with or
without redundancy
• Two redundancy
configurations available
– N+1(top)
– N+N (bottom)

Rev. 6.21 HP Restricted 17

Power supply LEDs
• Power supply has 2 LEDs
• LED locations shown in
the graphic
• LED indicator
combinations described in
the table

1 Power LED 2 Fault LED

Condition
(green) (amber)
No AC power to the power
Off Off
supply
On Off Normal

Off On Power supply failure

Rev. 6.21 HP Restricted 18

Enclosure cooling (1 of 2)
• c-Class enclosure uses new
high air flow, high pressure fan
design
• Supports up to 10 fans
• Delivered with 4 fans
• Minimum of 4 fans required to
cool
– Enclosure
– 1 full-height blade or up to 2
half-height blades

Rev. 6.21 HP Restricted 19

Enclosure cooling (2 of 2)
• Removal of a fan unit automatically closes louvers in the
enclosure to maintain proper airflow
• Installation of a fan unit automatically opens louvers in the
enclosure to allow airflow from that fan unit to the enclosure

Rev. 6.21 HP Restricted 20

PARSEC architecture
• Advantages of both local
and central cooling
• Blades divided into 4 Fan 1 Fan 2 Fan 3 Fan 4 Fan 5
zones
• Fans in each zone provide Blade Blade Blade Blade
Zone 1 Zone 2 Zone 3 Zone 4
– Cooling for blades in that
zone 2 FH or 2 FH or 2 FH or 2 FH or
4 HH 4 HH 4 HH 4 HH
– Redundant cooling for
other blades
Fan 6 Fan 7 Fan 8 Fan 8 Fan 10
• If a fan fails, other fans
are directed to run faster
to maintain the proper PS 1 PS 2 PS 3 PS 4 PS 5 PS 6
cooling capacity

Rev. 6.21 HP Restricted 21

Cooling scalability
• Install the number of fans required in
the locations shown
– 4 fans – Cool up to 1 FH-blade or 2
HH-blades
– 6 fans – Cool up to 4 FH-blades or 8
HH-blades
– 8 fans – Cool up to 8 FH-blades or 16
HH-blades
– 10 fans – Provides optimum power
usage and acoustics

Rev. 6.21 HP Restricted 22

Fan LED
• Fan unit has single status
LED
– LED position is circled on
the graphic
– LED indicators are
described in the table

LED color Fan status

Solid green Fan working normally
Solid amber Fan failure
Blinking amber Check the front LCD for information

Rev. 6.21 HP Restricted 23

Thermal Logic technologies (1 of 2)
• Integrated thermal monitoring and management within the
Onboard Administrator program
• Technologies
– Instant thermal monitoring
• Real-time heat, power, and cooling data
– Thermal Logic Air Movers
• Control algorithm to optimize air flow, acoustics, power, and
performance
– PARSEC architecture
• Parallel, redundant, and scalable airflow cooling design

Rev. 6.21 HP Restricted 24

Thermal Logic technologies (2 of 2)
– Pooled power
• N+N power redundancy
– Dynamic Power Saver
• Power load shifting for max efficiency and reliability
– Power regulator
• iLO-controlled speed stepping (for blade server processors)
– Power workload balancing
• Virtualization to maximize performance/watt
• Monitored through Onboard Administrator management
display (see next slide)

Rev. 6.21 HP Restricted 25

Thermal Logic display
Rack-level
Enclosure BTU/Hr.
Inflow &
Outflow
Temp.

Actual Power
Utilization

Max Available
Power

Rev. 6.21 HP Restricted 26

Supported
blades

Rev. 6.21 HP Restricted 27

Supported server blades
• c-Class BladeSystem supports
– half-height blade server
• BL460c
− Intel XEON based
− Two Embedded NICs
− Two Mezzanine card slots
− Option for two hot-pluggable SFF SAS or SATA drives

– full-height blade server

• BL480c
− Intel XEON based
− Four Embedded NICs
− Three Mezzanine card slots
− Option for four hot-pluggable SFF SAS or SATA drives

– Other future full-height and half-height blade models

Rev. 6.21 HP Restricted 28

Storage blade
• HH blade with six drive slots
– Hot-pluggable SFF SAS or SATA drives
• SAS – 10K rpm, 36GB and 72GB
• SATA – 5.4K rpm, 60GB
• Must be placed in a blade slot to the right of the supported
server blade
• Max of 15 storage blades per enclosure

Rev. 6.21 HP Restricted 29

Interconnect
devices

Rev. 6.21 HP Restricted 30

c-Class interconnect options
• 8 interconnect bays in c7000 enclosure
• Switches/pass-throughs integrated into c-Class
BladeSystem
– Support up to 4 independent fabrics simultaneously in c7000
– Switches provide “zero-footprint” fabric connectivity
• c7000 supports single-wide and double-wide modules
– Bays 1 & 2
• Support Ethernet modules only (which connect to embedded
NICs)
– Bays 3 - 8
• Support Ethernet or Fibre Channel switches and pass-throughs
• Can support other fabric products as they are developed

Rev. 6.21 HP Restricted 31

c7000 Interconnect bays

LAN switch or pass through LAN switch or pass through

Fabric switch or pass through Fabric switch or pass through

Fabric switch or pass through Fabric switch or pass through

Fabric switch or pass through Fabric switch or pass through

Rear view of
enclosure

Rev. 6.21 HP Restricted 32

Ethernet pass-through module
• 16 patch panel ports
– 1:1 NIC to patch panel
– Non-blocking
• Full-duplex 1Gb Ethernet
only
– No auto-negotiation for
10/100/1000
• No LAN management required inside the enclosure
• Basic status monitored by the enclosure manager
• Second pass-through device required for redundant
connectivity

Rev. 6.21 HP Restricted 33

GbE2c Ethernet blade switch
• BNT (Nortel) Ethernet switch
• Designed specifically for
c-Class BladeSystem HP BladeSystem Ethernet Module

• Based on p-Class GbE2

• Same Layer 2 features as found in the GbE2 3.X firmware
• Ports
– 16 internal downlinks
– 5 RJ-45 external uplinks
– 2 internal cross-connects

Rev. 6.21 HP Restricted 34

Catalyst Blade Switch 3020
• Cisco Systems Ethernet switch designed for c-Class
BladeSystem
• Same Layer 2 features as found in the p-Class CGESM
• Integrated copper uplinks and optional SFPs for fiber
support
• Ports
– 16 internal downlinks
– 8 uplinks
• 8 copper uplinks
• 4 SFP Fibre Channel ports can be substituted for 4 copper ports
– Ports 23 and 24 can be configured as internal cross connects 17x

17 Cisco Catalyst Ethernet Blade Switch 3020 17 21

23x
SYS
T
STA
T
DLX
SPD
MOD 20 20 24
CONSOLE UI 18 24x
E 17 19 20
D 18x

Rev. 6.21 HP Restricted 35

SAN pass-through module
• 16 fully-independent
FC pass-through ports
• Kit contains 1 module
with 4 Gb SFPs installed
– Order 2 kits for redundant
fabric connections
• Supports auto-negotiated
fabric speeds
– 1, 2, and 4Gb connections
– Independent fabric speed for each port
• iLO link for management
• Interoperates with industry leading SAN environments

Rev. 6.21 HP Restricted 36

SAN switch
• Brocade 4Gb SAN switch for HP c-Class
BladeSystem
• Three configurations
– 4/24 SAN Switch (pictured)
• 16 4Gb internal ports
• 8 4Gb external ports
– 4/12 SAN Switch (Dynamic Ports on Demand)
• 8 4Gb internal ports (default configuration)
• 4 4Gb external ports (default configuration)
• Flexible port license assignment
• 12 Port Upgrade License available
– 4/24 SAN Switch Power Pack
• Detailed switch discussion later in this training

Rev. 6.21 HP Restricted 37

Ports on Demand

Today:
24 16
23 15
• Static 1:1 relationship between blade slots, and
8
22 14 internal switch ports
7
21 13 6 • Specific slots must be populated to match with a
20 12 5
Pre-reserved corresponding active port
19 11 4 • Significantly enhances usability for low-touch
18 10 3
17 9
server customers
2
Internal 1  Dynamic Ports on Demand (DPOD)
– Any server slot can be populated
– Ports allocated on first-come-first serve basis to
any location, including external ports
– N # of pre-reserved ports decrements N # of
ports from the POD pool
– Removing a server, or ext. port and releasing
that port increments available POD pool
– Works with existing license keys
Rev. 6.21 HP Restricted 38
Ports on Demand cont.
Dynamic POD
Port Count Base = 12 ports
Double License = 24 ports

Port Distribution Base = 12 ports in any mix

Double = 16 internal ports
8 external ports

Rev. 6.21 HP Restricted 39

BladeSystem
c7000 Fabric
Architecture

Rev. 6.21 HP Restricted 40

Embedded NICs and mezzanine cards
• Embedded NICs in the server blades
– One dual-port NIC for half-height blades
– Two dual-port NICs for full-height blades
– Embedded NICs associate with interconnect bays 1 & 2
• Mezzanine cards are attached to the server blades
– Two mezzanine card slots on the half-height blades
– Three mezzanine card slots on the full-height blades
• Inserting the server blade establishes connections between
embedded NICs/mezzanine cards and the enclosure
midplane
• Midplane circuitry routes signals from NICs/mezzanine
cards to interconnect devices and vice versa

Rev. 6.21 HP Restricted 41

Port mapping
• Onboard Administrator
– Administers the fabric interconnect rules of the enclosure
• Includes electronic keying to analyze the enclosure fabric
configuration for compatibility
• May prevent switch device power-up if fabric conflicts are present
• May disable switch ports due to fabric connection conflicts
– Provides mapping information via Web-based interface and
GUI

Rev. 6.21 HP Restricted 42

Sample HH blade port mapping, c7000
• Mezzanine Slot 1 supports single or dual port cards
• Mezzanine Slot 2 supports single, dual or quad port cards

Rev. 6.21 HP Restricted 43

Sample FH blade port mapping, c7000
• All three Mezzanine Card Slots supports single, dual or
quad port cards

Rev. 6.21 HP Restricted 44

Management

Rev. 6.21 HP Restricted 45

Management options
• Several management options available
– Onboard Administrator
– HP Systems Insight Manager
– Selected Essentials products
– iLO
• Options permit local or remote management of the
enclosure
• Simplified management experience
– Single installation for all components
– All components tested together
– Single license for all components
– Single security model for the entire environment

Rev. 6.21 HP Restricted 46

Onboard Administrator (1 of 2)
• Enclosure-resident management capability
• Roles
– Monitors and manages elements of the enclosure
• Shared power
• Shared cooling
• I/O fabric
• iLO
– Provides secure interface
– Integrates with all HP systems management applications
• Provides local and remote management capability
– Front LCD panel (aka Insight Display) allows
installation/configuration actions at the rack
– Browser access permits remote management through GUI

Rev. 6.21 HP Restricted 47

Onboard Administrator (2 of 2)
• Front LCD panel provides
easy installation and
diagnostics
• Uses installation and
setup wizards
Front panel LCD

Enclosure
links
Ethernet and
serial connectors

Redundant Onboard
Administrator Modules

Rev. 6.21 HP Restricted 48

Onboard Administrator module
• Module runs the Onboard
Administrator application
• Single module required
for enclosure
management
– Space available for
redundant module
• Provides Ethernet and
serial connector

Rev. 6.21 HP Restricted 49

Front LCD panel
• LCD contains display area and
navigation buttons
– Arrow buttons move cursor
up, down, left, and right
– OK button executes
commands
• LCD background color indicates
enclosure status
– Blue – Chassis UID
– Amber – Error condition
– Green – Normal operations
– White – Configuration/setup
– Cyan – Check display for
message
• Pushing down-arrow presents
additional main menu options
Rev. 6.21 HP Restricted 50
Enclosure settings
• To view enclosure settings
from main menu
– Press down-arrow button
once
– Press OK button
– Enclosure Settings display
appears

Rev. 6.21 HP Restricted 51

LCD diagnostics
• Error condition display could
look like the sample display
• Selecting the Fix THIS tab
activates a wizard that
describes the problem and
provides step-by-step
instructions for fixing it
• Refer to the Onboard
Administrator user guide for
additional information on
Onboard Administrator

Rev. 6.21 HP Restricted 52

OA “First Power Up” Checks
• Inventory installed power supplies and fans
• Perform location check on all power supplies and fans
• Turn on power supplies, perform fan speed checks
• Inventory and turn on all installed I/O modules
• Wait for wizard to be completed by installer
• Use power config settings to determine max available power
• Subtract power needed for fans and interconnect modules
• Begin server blade power/cooling/location/configuration
checks
– If checks OK, permit server blade to power up
– If error detected, deny power up, send alert, display error on
LCD
– Continue checks and power-up with next server blade
Rev. 6.21 HP Restricted 53
Sample OA Setup Wizard GUI display

Rev. 6.21 HP Restricted 54

Sample health status display

Overall status

Individual
device status

Rev. 6.21 HP Restricted 55

Product
overview –
4Gb SAN
switch

Rev. 6.21 HP Restricted 56

Brocade 4Gb SAN Switch
for HP c-Class BladeSystem (1 of 5)

Handle Connector
Handle Latch

Reset
Button

Status External Port

LEDs FC ports LEDs

Rev. 6.21 HP Restricted 57

Brocade 4Gb SAN Switch
for HP c-Class BladeSystem (2 of 5)
Delivers embedded Fibre Channel SAN technology to the new
HP c-Class BladeSystem
Value
• Best-in-class performance and value
• Reduces need for costly components
• Improved utilization of existing storage and BladeSystem assets,
increased administrator productivity, lower cost of management
Simplicity
• Rapid deployment with minimal configuration
• Little or no management required in small fabric configurations
• Easy growth and integration into all industry leading SAN environments
Agility
• Support for modular scalability
• Single code stream for forward and backward compatibility with entire
Brocade and HP StorageWorks B-series product lines
• Largest installed base and most widely tested interoperability

Rev. 6.21 HP Restricted 58

Brocade 4Gb SAN Switch
for HP c-Class BladeSystem (3 of 5)
Extending tangible savings for business
Conventional IT
Capital Expenses Operational Costs

Bladed Lower costs, greater agility

People • Reduce or eliminate lengthy

Costs processes
People • Simplify procedures
• Reduce complexity
Costs
Data
Center •Lower power, fewer cables
Costs •Less space
Data
Center
3rd party Costs • Reduced Ethernet/FC costs
3rd party • Lower cost HP management tools

HP HP
servers servers • Integration of components
& & • Sharing of resources
storage storage • Infrastructure control software

Rev. 6.21 HP Restricted 59

Brocade 4Gb SAN Switch
for HP c-Class BladeSystem (4 of 5)

Why Brocade 4Gb SAN Switch for HP c-Class

BladeSystem Fibre Channel connect?
– Superior performance
– Redundant FC switching for HP c-Class BladeSystem
– Industry-best FC installed base with which to interoperate
– Industry leading 4Gb/sec FC performance in a modular form
factor
Flexible design allows entry to enterprise SAN
connectivity
– Best value for FC embedded technology in the industry
– Lowest cost and easiest to manage FC interconnect solution
for HP c-Class BladeSystem

Rev. 6.21 HP Restricted 60

Brocade 4Gb SAN Switch
for HP c-Class BladeSystem (5 of 5)

• Enclosure supports Brocade 4Gb SAN switch for HP c-

Class
BladeSystem
• Embedded in c-Class enclosure
• Fully compatible with HP B-Series
– Can operate as only switch in the fabric or with other switches
• Supports heterogeneous fabrics
• Hot-pluggable

Rev. 6.21 HP Restricted 61

SAN Switch features (1 of 2)
• Power and cooling provided by the c-Class enclosure
• Switch insertion into enclosure makes midplane connections
for power, management and internal ports
• Identification to HP chassis management via HP-specified
SEEPROMs
• Internal ports
– Auto-negotiation of link speed ( 1 Gb, 2Gb or 4Gb/sec) to
highest common speed for each port independently
– Universal and self-configuring ports capable of becoming
F_ports

Rev. 6.21 HP Restricted 62

SAN Switch features (2 of 2)
• External ports
– Auto-negotiation of link speed (1, 2, or 4Gb/sec) to highest
common speed
– Compatible with both short- and long-wavelength SFPs
– Universal and self-configuring ports (F_port, FL_port, or
E_port)
• Managed locally or remotely
– Onboard Administrator
• Using Insight Display (front LCD panel)
• Browse to the OA GUI
– System Insight Manager

Rev. 6.21 HP Restricted 63

External ports
• 8 Fibre Channel ports
• External ports numbered 17-20 and 21-0
– Ports 1-16 are reserved for internal ports to the blades

Item number Description

1 Left bank – Ports 17, 18, 19, and 20

2 Right bank – Ports 21, 22, 23, and 0

Rev. 6.21 HP Restricted 64

SAN switch LEDs
• All SAN switch
LEDs are located
on the port side of
the switch
• Each port has a
port status and port
speed LED
Item
Name Description
• LEDs are shown in Number

the graphic and 1 Unit ID LED (UID) Blue or off

2 Health ID LED Amber or green
described in the
Module Status
table 3
LED
Amber or green

4 Port Status LED Amber or green

5 Port Speed LED Off, amber, or green

Rev. 6.21 HP Restricted 65

Unit ID LED
• Used to locate the specific switch
• Indications
– Off – Normal operation. No active ID request
– Blue – Lights steady blue when an ID request is active for that
switch

Rev. 6.21 HP Restricted 66

Health ID LED
• Bi-color LED
– Green
– Amber
• Indications
– Off – Power is off to the switch
– Green – Power is on. The switch is operating normally
– Amber – Power is on. There is a fault in the switch.

Rev. 6.21 HP Restricted 67

Module Status LED
LED name LED color Status of hardware Recommended action
Switch is off, boot is not Verify the switch is on
No light
complete, or boot failed and boot is completed.
Steady green Switch is on and functioning No action required.
Check environmental
One or both of the following
is true: conditions, error log,
Module Status Port Status LEDs,
Flashing green • One or more environmental
LED SFPs, cables, and
(on 1 second, ranges are exceeded
loopback plugs. Correct
off 1 second) •Error log contains one or error condition. Clear
more port diagnostic error error log. Rerun
messages diagnostics to verify fix.
Boot-up state; one or more
Amber Needs attention.
ports offline

Rev. 6.21 HP Restricted 68

Port Status LED (1 of 2)
LED name LED color Status of hardware Recommended action
No light or signal carrier
No light (transceiver or cable) Check transceiver and cable.
detected.
Port is online but has no
Steady green No action required.
traffic.
Slow-flashing Port is online but Verify the correct device is
green (on 1 segmented, indicating a connected to the port and
Port Status
second, off 1 loopback cable or the switch and port settings
LED
second) incompatible switch. are correct.
Fast-flashing
green (on ½ Port is in internal loopback
No action required.
second, off ½ (diagnostic).
second)
Flickering Port is online with traffic
No action required.
green flowing through it.

Rev. 6.21 HP Restricted 69

Port Status LED (2 of 2)
LED name LED color Status of hardware Recommended action
Port is receiving light or
Steady amber signal carrier, but is not yet No action required.
online.
Port is disabled as a result of Enable the port using the
Slow-flashing diagnostics or portDisable portEnable command. If the
amber (on 1 command. If the LEDs for all LEDs for all ports are
second, off 1 ports are slow-flashing slow-flashing amber, enable
second amber, the switch could be the switch by entering the
Port Status disabled. switchEnable command.
LED Check the port status LEDs,
Fast-flashing error log, transceiver, and
amber (on ½ cable or loopback plug. Clear
Port is faulty.
second, off ½ the error log. Rerun the
second) diagnostics to verify the error
condition is fixed.
Alternating
Check configuration of the
amber and Port is bypassed.
Fibre Channel loop.
green

Rev. 6.21 HP Restricted 70

Port Speed LED

LED name LED color Description

Port is operating at
No light
1Gb/sec
Port is operating at
Port Speed LED Green
2Gb/sec
Port is operating at
Amber
4Gb/sec

Rev. 6.21 HP Restricted 71

Reset button
• Reset button located to the left of the status LEDs
• Recessed micro switch
– Use a pin or similar size object to push the switch
• Used to reset/reboot under the following conditions
– Switch not responding

Rev. 6.21 HP Restricted 72

Installing the
switch

Rev. 6.21 HP Restricted 73

Pre-installation considerations (1 of 2)
Electrical
• Switch requires 35 Watts of power from the enclosure
• Each server to be connected to the switch requires a Fibre
Channel mezzanine card (one card connects to 2 bays)
Environmental
• Verify air vents on the enclosure are not blocked or
restricted
• Verify the ambient temperature at the front of the enclosure
does not exceed 35 degrees Centigrade (95 degrees F)
• Verify that SFP dust cover is installed in each unoccupied
SFP slot.

Rev. 6.21 HP Restricted 74

Pre-installation considerations (2 of 2)
• Single-wide switch device deployment:
– Fabric designation is the same for both Bays in a “pair”.
– I/O Bay pairs are 1 & 2, 3 & 4, 5 & 6, 7 & 8.
• Populate Fibre Channel I/O blades and Ethernet I/O blades in pairs.
– I/O Bays 1 & 2 are always an Ethernet pair.
– A single I/O Bay occupied with a blank installed to the left or right is valid.
– A single switch with a fabric-matched pass-through module is valid.
– Two fabric-matched switches or two fabric-matched pass-throughs is valid.
– Different fabrics (Fibre Channel and Ethernet) can never be mixed in an I/O
Bay pair.
• Ethernet Pass-Through or any Ethernet Switch or any combination
thereof: I/O Bays 1-8 possible with the appropriate server & mezzanine
card combination.
• Fibre Channel Pass-Through or any Fibre Channel Switch or any
combination thereof: I/O Bays 3-8 possible with the appropriate server
& mezzanine card combination.

Rev. 6.21 HP Restricted 75

Installing the switch (1 of 3)
• The switch is a hot-pluggable device, so enclosure can be
either on or off during switch installation
• Only one switch should be installed and set up at a time in
order to avoid duplicate default IP addresses.
• Use the following procedure
1. Determine which I/O bay should receive the switch.
– This is determined by FC mezzanine card location and
enclosure port mapping
– All FC mezzanine cards connecting to the switch must be in the
same server mezzanine slot. (example: mezzanine slot 1)

! Important: Populate all enclosure I/O bays with a switch, a pass-through module, or
one of the blanks provided with the enclosure. Populate and set up only one FC
switch at a time to avoid IP address conflict with duplicate default addresses.

! Important: Populate all external switch ports with either an SFP or SFP dust plug..
Keep optical port dust plugs in the SFPs until an optical cable is connected. This
action improves cooling and prevents contamination.

Rev. 6.21 HP Restricted 76

Installing the switch (2 of 3)
Caution! Use appropriate electrostatic discharge (ESD)
techniques when handling the switch.

2. Move the handle latch to the right to release the installation

handle.

Rev. 6.21 HP Restricted 77

Installing the switch (3 of 3)
3. Insert the switch into the enclosure I/O bay and push it
firmly into place.

4. Press the installation handle into the latch to lock the

switch in place

Rev. 6.21 HP Restricted 78

Installation verification
• With the switch installed, Onboard Administrator verifies
switch type matches mezzanine cards in servers
– If devices match, OA permits the switch to power up
– If devices do not match, OA does not allow the switch to power
up
– If switch does not power up, check the enclosure and switch
status in the OA GUI. The OA GUI virtual power button must
be on in order to allow the switch to power up.
• Verify the following LED indications are present
– UID – Off
– Health ID – Steady green light
– Module Status – Steady green light

Rev. 6.21 HP Restricted 79

OA GUI front and back view

Rev. 6.21 HP Restricted 80

OA Switch Status and Virtual Button tab

Rev. 6.21 HP Restricted 81

OA Switch Power on and Off

Rev. 6.21 HP Restricted 82

Switch GUI (http://ip address of switch)

Rev. 6.21 HP Restricted 83

Configuring
the switch

Rev. 6.21 HP Restricted 84

Setting the IP address (1 of 4)
Caution! Do not connect the switch to the LAN network until the IP address
is correctly set.

• IP address is set using a serial connection

– Workstation/server must have a terminal application
• HyperTerminal in a Windows environment
• TERM in a UNIX environment
– Null modem serial cable
• Obtain IP address and related information from network
administrator
• Use the following procedure to set the IP address

Rev. 6.21 HP Restricted 85

Setting the IP address (2 of 4)
1. Verify there is power to the enclosure.
2. Identify the active Onboard Administrator module.
3. Connect the serial cable to the workstation/server and
active OA module.
4. Configure terminal application settings as follows
a. Windows environment
1) Bits per second – 9600
2) Databits – 8
3) Parity – None
4) Stop bits – 1
5) Flow control – None
b. UNIX environment, enter: tip /dev/ttyb -9600

Rev. 6.21 HP Restricted 86

Setting the IP address (3 of 4)
5. Log on to Onboard Administrator.
6. Identify the bay number for the installed switch.
7. At the command line, enter
a. connect interconnect x
(where x is the interconnect bay that the
switch is installed in.)
b. User: admin
c. Password: password

Note: The commands are case sensitive, so should be entered as shown.

Rev. 6.21 HP Restricted 87

Setting the IP address (4 of 4)
8. At the command line, enter: ipaddrset.
9. Enter the remaining IP addressing information, when
prompted.
10. Optionally, verify the IP address is correct by entering
ipaddrshow at the command prompt.
11. Record the IP addressing information and store in safe
place.
12. Enter Exit and press Enter to log out of the terminal
session.
13. Cuando le cambias el password se lo debes cambiar igual para todos los modulos y se sale
de el con ctrl + _ (undescor)

14. Disconnect the serial cable.

15. Plug in the next switch into the enclosure and set that
switch up. Only one switch should be installed and set up
Rev. 6.21 at a time in order to avoid duplicate default IP addresses.88
HP Restricted
Items required for configuring switch
• Switch installed in an enclosure
• IP address, subnet mask, and gateway address recorded in
Setting the IP Address procedure
• Ethernet cable
• SFP transceivers and compatible optical cables
• Optional – Access to an FTP server
– Used to backup the switch configuration

Rev. 6.21 HP Restricted 89

Connecting to the CLI (1 of 2)
Create an Ethernet connection
1. Locate the iLO port on the OA module.
2. Connect the Ethernet cable to the iLO port on the OA
module and the workstation/server or to the network
containing the workstation.

! Important: Verify the switch is not being modified from any other
connections during the remaining steps.

3. Open a telnet connection using the IP address of the

switch set earlier (the login prompt displays when the
connection locates the switch in the network).

Rev. 6.21 HP Restricted 90

Connecting to the CLI (2 of 2)
4. Enter the user name, using the administrative account:
admin.
5. Enter the password. Default password is password.
If passwords have not be changed from default value, you
are prompted to change them. Do one of the following:
1) Enter new system passwords.
2) Press Ctrl+C to skip the password prompts.

Note: Up to two simultaneous admin sessions and four user sessions can be created.

6. Verify the login was successful (successful login displays

switch name and user ID to which you are connected).

Rev. 6.21 HP Restricted 91

Setting the time
• Date and time are used for logging events
• Switch operation not dependent on date and time setting
– Switch with incorrect time still operates properly
• Use the following procedure to set the date and time
1. Connect to the switch and log in as admin.
2. Enter the CLI date command using the following syntax
date “mmddHHMMyy”.
• To switch time zones, refer to the tsTimeZone command
in the Brocade Fabric OS 5.x Command Reference Guide

Rev. 6.21 HP Restricted 92

Disabling/enabling a switch (1 of 2)
• By default, a switch is enabled following completion of
POST
• Switch can be disabled or enabled, as required
To disable
1. Connect to switch and log in as admin.
2. Enter the command: switchDisable.
All FC ports on the switch are taken offline. If the switch
was part of a fabric, the fabric reconfigures.

Rev. 6.21 HP Restricted 93

Disabling/enabling a switch (2 of 2)
To enable
1. Connect to the switch and log in as admin.
2. Enter the command: switchEnable.
All FC ports that pass POST and OA fabric match (see
below) are enabled. If the switch has interswitch links (ISLs)
to a fabric, it joins the fabric.
Keep in mind that the OA forces a disable of any port in a
configuration where there is a mismatch between the type of
mezz card and the switch. (Ethernet NIC in a mezz card slot
connecting to a FC switch). This disable will not be over-
ridden by a switch enable command.

Rev. 6.21 HP Restricted 94

Enabling/disabling a port
To enable a port
Keep in mind that the OA will disable a port if there is a
mismatch between the type of mezz card and the switch.
(Ethernet NIC in a mezz card slot connecting to a FC switch).
This disable cannot be over-ridden by a port enable command
on the switch.
1. Connect to the switch and log in as admin.
2. Enter the command: portEnable <portnumber>,
where portnumber is the number of the port to be enabled.
To disable a port
1. Connect to the Switch and log in as Administrator.
2. Enter the command: portDisable <portnumber>,
where portnumber is the number of the port to be
disabled.
Rev. 6.21 HP Restricted 95
Dynamic Ports on Demand (1 of 4)
Brocade 4/12 SAN Switch
• Includes Dynamic Ports on Demand with 8 internal ports
enabled and 4 external ports enabled (default
configuration).
• Any 12 ports on the switch can be used.
• Ports are assigned to a license only if they appear ONLINE
to the switch (i.e. an operating FC mezzanine card or active
external link is discovered)
• Licensed port count that is not used on initial installation will
be allocated as new ports are discovered when they come
ONLINE during operation
• Once a license has been assigned to a port, that
association remains until released by explicit operator action
Rev. 6.21 HP Restricted 96
Dynamic Ports on Demand (2 of 4)
Brocade 4/12 SAN Switch
• A license can be released from a port and returned to the
unassigned license pool.
– Use CLI command “licensePort –release portnum” to release
• A license reservation can be entered for OFFLINE ports to
ensure that a particular port will be able to come ONLINE in
the future
– Use CLI command “licensePort –reserve portnum” to reserve a
future license assignment for a specific port
• There are equivalent webtools operations for the
licensePort release and reserve operations

Rev. 6.21 HP Restricted 97

Dynamic Ports on Demand (3 of 4)
Brocade 4/12 SAN Switch
• The POD method used in a product (static or dynamic) must
be configured at the time of manufacture via the CSD
process.
• The CLI command “licenseport --method dynamic” changes
the POD method to dynamic.
– Any of the 24 ports will be enabled to be F ports or E ports up
to the limit of the number available in the unallocated un-
reserved POD pool.
– The switch must be rebooted for the method change to take
effect.
– There is no support for on-site conversion of a unit from using
the dynamic POD method to static port method.

Rev. 6.21 HP Restricted 98

Dynamic Ports on Demand (4 of 4)
Brocade 4/12 SAN Switch
• The Switch is shipped with 4 pre-reserved external ports.
– These reserved POD licenses can be released using the
“licensePort –release portnum” command
• Use CLI command “licenseport –show” shows what POD
licenses are installed, what ports are reserved, and what
ports are allocated.
• Use CLI command “help licensePort” for more information
from the switch on the licensePort command.

Rev. 6.21 HP Restricted 99

Changing the FC domain ID (1 of 2)
To modify the domain ID
1. Disable the switch by entering: switchDisable.
2. Enter Configure, then enter a new value or press Enter
to accept default value.
3. At the fabric parameters prompt, enter Y and press Enter.
4. Enter a unique domain ID (1 through 239).
5. Complete the remaining prompts or press Ctrl+D to
accept the remaining default values.

Rev. 6.21 HP Restricted 100

Changing the FC domain ID (2 of 2)
6. Re-enable the switch using the command:
switchEnable.
7. Confirm changes made to the domain ID by entering:
fabricShow.
8. Optionally, verify switch policy settings and specify any
custom status policies that need to change.
a. Enter switchStatusPolicyShow to verify the current
policy settings. If desired, change policy settings by
entering switchStatusPolicySet at the prompt. Refer
to the HP StorageWorks Fabric OS 5.x command
reference guide for available parameters.
b. Customize the status policies, as required.
9. To deactivate the alarm for a particular condition, enter 0
at the prompt for that condition.
Rev. 6.21 HP Restricted 101
Verifying the configuration
Verify the configuration by doing the following
• Check LEDs to verify that all components are functional
• Enter switchShow to get information about switch and port
status
• Enter fabricShow to get information about the fabric

Rev. 6.21 HP Restricted 102

Backing up the configuration
• HP recommends regular backups
– Ensures that a recent configuration is available for download to
replacement switches
• Back up the configuration to an FTP server by entering
configUpload and following the prompts

Rev. 6.21 HP Restricted 103

Managing
the switch

Rev. 6.21 HP Restricted 104

Management tools
• OA controls port enable and disable.
• Tools available within switch firmware are
– Advanced Web Tools
– Standard SNMP applications
– Management Server
– Command Line Interface
• Tools are used to monitor
– Fabric topology
– Port status
– Physical status
– Information used for performance analysis and system
debugging

Rev. 6.21 HP Restricted 105

Advanced Web Tools
• GUI that enables administrators to monitor and manage
single/small fabrics, switches, and ports from standard
workstation
• Provides control point for:
– Advanced Zoning
– ISL Trunking
– Advanced Performance Monitoring
– Fabric Watch
– Remote Switch
• Provides interface to telnet commands to perform functions
and diagnostics available only through telnet interface
• Refer to Brocade Fabric OS 5.x Advanced Web Tools
Administrator Guide for detailed information
Rev. 6.21 HP Restricted 106
Monitoring switch events (1 of 2)
• Advanced Web Tools displays switch name, message
number, time stamp, whether event is a logical or hardware
event, severity level, message identifier, and detailed error
message
• Severity levels are:
– Critical – Serious problem that can cause partial or total
subsystem failure
– Error – Condition that does not significantly affect overall
system functionality
– Warning – Indicates condition that should be checked before
failure occurs
– Information – Reports current non-error status of system

Rev. 6.21 HP Restricted 107

Monitoring switch events (2 of 2)
• From Switch Events window, clicking Filter button permits
user to define which event messages are displayed
• Display automatically updates every 15 seconds (no manual
refresh option is available)
• Refer to Brocade Fabric OS 5.x Advanced Web Tools
administrator guide for detailed information and procedures

Rev. 6.21 HP Restricted 108

Monitoring switch status (1 of 3)
• Advanced Web Tools provides a switch status report
– Select a switch from the Fabric Tree (Switch View displays)
– Click the Status button on Switch View (detailed switch health
report displays – see next slide)
– Optional: Click the underlined links in the left panel to display
detailed information about ports and Switch Availability
Monitoring (SAM)

Rev. 6.21 HP Restricted 109

Monitoring switch status (2 of 3)

Rev. 6.21 HP Restricted 110

Monitoring switch status (3 of 3)
• Optional: Mouse-over the
Action box and click an
action to
– Refresh the information
– Customize the report
– View the data in raw XML
format
– View the style sheet for the
report
– View the XML schema for
the report

Rev. 6.21 HP Restricted 111

Monitoring port status (1 of 2)
• Switch View displays port graphics with blinking LEDs
• One LED indicates port status, the other port speed
• Background color of port icon indicates port status
– Green – Healthy
– Yellow – Marginal
– Red – Critical
– Gray – Unmonitored
• Completely blue port icon indicates port is buffer-limited
• Ports that are grayed out are not licensed

Rev. 6.21 HP Restricted 112

Monitoring port status (2 of 2)
• A Port Information
display can be
generated by
clicking on a port
in the left portion
of the Switch View
display

Rev. 6.21 HP Restricted 113

Trace dump (1 of 2)
• Snapshot of switch behavior
• Used by developers and troubleshooters to understand
what is contributing to a specific behavior
• Tracing is always on
– Trace information is placed into a circular buffer in system
RAM
– Buffer is periodically frozen and saved
– Saved data is called a trace dump
• Trace dump generated when
– Triggered manually (using the traceDump command)
– A critical-level LOG message occurs
– A particular LOG message occurs (user-defined trigger)
– A kernel panic occurs
– The hardware watchdog timer
Rev. 6.21 expires
HP Restricted 114
Trace dump (2 of 2)
• Trace dump is maintained until either
– It is uploaded to the FTP host
• Can be uploaded manually or automatically
– Another trace dump is generated
• If second trace dump is generated before the first is uploaded, the
first is overwritten
• Use the Trace tab of the Switch Admin module to
– View and configure the trace FTP host target
– Enable or disable automatic trace uploads
– Manually upload a trace dump
• Refer to Brocade Fabric OS 5.x Advanced Web Tools
administrator guide for further details and procedures

Rev. 6.21 HP Restricted 115

Managing zoning (1 of 3)
• Zoning used to partition SAN into logical groups of devices
that can access each other
• Zone characteristics include:
– They can be configured dynamically and vary in size
– Devices can belong to more than one zone
– Zone members can access only other members of the same
zone
• Devices outside the zone cannot access devices within the zone
and vice versa
• When using a mixed fabric – switches with different
firmware versions – use the switch with highest version
number to perform zoning tasks

Rev. 6.21 HP Restricted 116

Managing zoning (2 of 3)
• Zoning is monitored and managed using the Advanced Web
Tools Zone Admin module
– Zone Admin icon appears on Fabric Toolbar only if an
Advanced Zoning license is installed on the switch
• Zoning information is collected from the selected switch
• User must be logged on as admin or switchAdmin to launch
Zone Admin module
– Admin has read and write privileges
– switchAdmin has read-only privileges
• Most zoning operations are disabled in read-only mode

Rev. 6.21 HP Restricted 117

Managing zoning (3 of 3)
• Zoning changes made using Zone Admin module must be
saved to become effective
– Changes are lost if Zone Admin module is closed before
changes are saved
– Saving – Updating zoning database on the switch with local
changes from the Advanced Web Tools buffer
– Refreshing – Copying current state of zoning database to the
Advanced Web Tools buffer, overwriting its current contents
• Refer to Brocade Fabric OS 5.x Advanced Web Tools
administrator guide for details of zoning requirements and
procedures

Rev. 6.21 HP Restricted 118

Managing ISL trunking (1 of 2)
• Trunking managed through the Trunking tab of the Switch
Admin module of Web Tools
• ISL trunking maintains in-order delivery
• Provides ability to:
– Display trunk group information
– Enable and disable trunking mode on a port
• License required for each switch participating in a trunk
• Trunks require cables of roughly equal length
– HP recommends a difference between cable lengths of no
more than 30 meters

Rev. 6.21 HP Restricted 119

Managing ISL trunking (2 of 2)
• Distributes traffic across shared bandwidth of all ISLs in a
trunking group
– Traffic flows through any available link in a group; not assigned
to a specific link
• Trunks are compatible with both short wavelength and long
wavelength fiber optic cables and transceivers
• Managed using either Fabric OS CLI commands or
Advanced Web Tools

Rev. 6.21 HP Restricted 120

Standard trunking criteria
• Must be a direct connection between participating switches
• Trunk ports must reside in same port group
• Trunk ports must run at the same speed (2Gb or 4Gb)
• Trunk ports must be set to the same ISL mode
• Trunk ports must be E_ports
• Cable lengths should differ by no more than 550 meters
• The switch.interopMode parameter must be set to 0
• Port ISL mode must be disabled (using the
portCfgIslMode command)
• Refer to the notes section of this slide for applicable
documentation

Rev. 6.21 HP Restricted 121

Related
information

Documentation
and Web sites
Rev. 6.21 HP Restricted 122
Documentation (1 of 2)
• HP StorageWorks Brocade 4Gb SAN Switch for HP c-Class
BladeSystem user guide, June 2006, PN AA-RWEBA-TE
• Brocade 4Gb SAN Switch for HP c-Class BladeSystem
installation instructions, June 2006, PN 5697-5678
• Pass-Thru installation instructions for HP c-Class
BladeSystem, April 2006, PN 413280-021
• HP BladeSystem c7000 Enclosure Setup and Installation
Guide, May 2006, PN 411272-001
• Brocade Fabric OS 5.x administrator guide Brocade Fabric
OS 5.x Advanced Web Tools administrator guide.

Rev. 6.21 HP Restricted 123

Documentation (2 of 2)
• Brocade OS 5.x command reference guide
• Brocade Fabric OS 5.x diagnostics and system error
messages reference guide
• Brocade Fabric OS 5.x Fabric Watch administrator guide
• Brocade Fabric OS 5.x MIB reference guide
• Brocade Secure Fabric OS administrator guide

Rev. 6.21 HP Restricted 124

Web sites
• Links to all of the listed Fabric OS 5.x documents are at:
http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.js
p?contentType=SupportManual&lang=en&cc=us&docIndexId=179111&t
askId=101&prodTypeId=12169&prodSeriesId=471548
• A link to the Brocade 4Gb SAN Switch documents is found
by going to www.hp.com and clicking on Servers and
clicking under Proliant on c-Class BladeSystem.
• A link to the Brocade 4Gb SAN Switch can also be found by
going to www.hp.com and clicking on Storage and clicking
on SAN Infrastructure and clicking on Fibre Channel
Switches and clicking under B-Series Fabric Entry Level on
Brocade 4Gb SAN Switch for c-Class BladeSystem.
Note: www.hp.com pages are subject to change.

Rev. 6.21 HP Restricted 125

Rev. 6.21 HP Restricted 127