#CLUS

Advanced Storage
Area Network
Design
Edward Mazurek
Technical Lead Data Center Storage
Networking
emazurek@cisco.com
@TheRealEdMaz
BRKSAN-2883

#CLUS
Agenda
• Introduction
• Design Principles
• Design Principles for Slow Drain
and Congestion Isolation
• Design Principles for SAN
Analytics
• Q&A

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Introduction
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session

How
1 Find this session in the Cisco Live Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKSAN-2883

by the speaker until June 18, 2018.

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Introduction
• Assumptions:
• Most SANs are reliable and have few problems
• Move from:
• Occasional problems that sometimes cause outages
• Move to:
• Infrequent problems and almost no outages

How can your SAN be more reliable, robust and less prone to errors?

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Introduction
• Cisco FC/FCoE SAN switches provide a host of advanced features
that can make your SANs more
• Robust
• Scalable
• Fault tolerant
• High performance
• Easy to Manage
• Easy to investigate / troubleshoot

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Design Principles
Design Principles
• VSANs
• Zoning and Smart Zoning
• N-Port Virtualization
• Trunking and Port-channeling
• MDS Internal CRC handling
• Device-alias
• SAN Security

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
VSANs
Introduced in 2002
• A Virtual SAN (VSAN) Provides a Method to Allocate Ports within
a Physical Fabric and Create Virtual Fabrics
• Analogous to VRFs in Ethernet Per Port Allocation
• Virtual fabrics created from larger cost-effective redundant
physical fabric
• Reduces wasted ports of a SAN island approach
• Fabric events are isolated per VSAN which gives further isolation
for High Availability
• FC Features can be configured on a per VSAN basis.
• ANSI T.11 committee and is now part of Fibre Channel standards
as Virtual Fabrics

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
VSANs
• Assign ports to VSANs

• Logically separate fabrics

• Hardware enforced

• Prevents fabric disruptions

• RSCN sent within fabric only

• Each fabric service (zone server, name server,

login server, etc.) operates independently in
each VSAN
• Each VSAN is configured and managed
independently

phx2-9513# show fspf vsan 43 vsan database

FSPF routing for VSAN 43 vsan 2 interface fc1/1
FSPF routing administration status is enabled vsan 2 interface fc1/2
FSPF routing operational status is UP vsan 4 interface fc1/8
It is an intra-domain router vsan 4 interface fc1/9
Autonomous region is 0
MinLsArrival = 1000 msec , MinLsInterval = 2000 phx2-9513# show zoneset active vsan 43
msec zoneset name UCS-Fabric-B vsan 43
Local Domain is 0xe6(230) zone name UCS-B-VMware-Netapp vsan 43
Number of LSRs = 3, Total Checksum = 0x00012848
#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
 Zoning & VSANs

VSAN 2 1. Assign physical ports to VSANs

2. Configure zones within each VSAN
Disk2
Disk3
Zone A Host1 Disk1
Zone C • A zone consists of multiple zone members
Zone B
Disk4 Host2
3. Assign zones to zoneset
Zoneset 1 • Each VSAN has its own zoneset

VSAN 3
4. Activate zoneset in VSAN
Zone A
Host4
• Members in a zone can access each other;
Zone B
Host3 Disk5 members in different zones cannot access
Disk6
each other
Zoneset 1

• Devices can belong to more than one zone

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
 Zoning examples
• Non-zoned devices are members of
the default zone
zone name AS01_NetApp vsan 42
member pwwn 20:03:00:25:b5:0a:00:06
• A physical fabric can have a member pwwn 50:0a:09:84:9d:53:43:54
maximum of 16,000 zones (9700-only
network)
device-alias name AS01
• Attributes can include pWWN, FC pwwn 20:03:00:25:b5:0a:00:06
alias, FCID, FWWN, Switch Interface device-alias name NTAP
fc x/y, Symbolic node name, Device member pwwn 50:0a:09:84:9d:53:43:54
zone name AS01_NetApp vsan 42
alias member device-alias AS01
member device-alias NTAP
• Recommended: Device-alias and/or
PWWN

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
The Trouble with sizable Zoning
All Zone Members are Created Equal
 Standard zoning model just Number of ACLs
has “members” 10,000

Number of ACL Entries

 Any member can talk to any 8,000
other member
6,000
 Recommendation: 1-1 zoning
4,000
 Each pair consumes two ACL
entries in TCAM 2,000

 Result: n*(n-1) entries per 0

zone

40
60
80
0
10
20
30
50
70
90
100
Number of Members
#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
 Smart Zoning

Operation Today – 1:1Operation

Zoning Today – Many
Operation
- Many Smart Zoning
8xI
Zones Cmds ACLs Zones Cmds ACLs Zones Cmds ACLs
4xT
Create 32 96 64
Create 1 13 132
Create 1 13 64
zones(s) zones(s) zones(s)
Add an +4 +12Add+8
an +1 Add+24
an +1 +8
initiator initiator initiator
Add a +8 +24 Add
+16
a +1 Add+24
a +1 +16
target target target

• Feature added in NX-OS 5.2(6)

• Allows storage admins to create larger zones while still keeping premise of single initiator & single target

• Dramatic reduction SAN administrative time for zoning

• Utility to convert existing zone or zoneset to Smart Zoning

#CLUS BRKSAN-2883 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15