J&K State Data Centre - Build Document

Build Document
For
Cisco Unified Computing System
VERSION CONTROL

Build Document for:

Endpoint Security - ApexOne
Document Number: 1 Version Number : 1.0
Effective Date: Review Date:

NAME TITLE Organization DATE

Author Nityanand L2-Compute CMSITS

Reviewer Ullas Project Manager CDAC
Authorizer Dr. Jayan V Joint Director CDAC

Accountability
Contact
Position: Name Dr. Manjeet Singh 9968888887
Number
Responsibility
Contact
Position: Name Project Manager
Number

Revision History
Sr.no Description Reviewer Date
1 Initial Version Ullas
2 Final Version Dr. Jayan V

Read By
NAME TITLE SIGNATURE DATE
Kavita Dawre Subject Matter Expert (SME)

Page 2 of 13 Confidential
TABLE OF CONTENTS

1. INTRODUCTION ...................................................................................................................................... 5

2. SYSTEM OVERVIEW ................................................................................................................................ 5

2.1 UCS 5108 BLADE CHASSIS .................................................................................................................. 5

2.2 BLADE SERVERS: ................................................................................................................................ 6

3 UCS 62XX SERIES FABRIC INTERCONNECTS .......................................................................................... 6

4 UCS MANAGER ........................................................................................................................................ 6

5 MAJOR COMPONENTS THAT COMPRISE UCS AT JK-SDC ....................................................................... 6

6 UCS CONFIGURATION INFORMATION .................................................................................................... 7

7 PHYSICAL CONNECTIVITY ...................................................................................................................... 7

8 VLAN/SEGMENT USAGE.......................................................................................................................... 7

8.1 UPLINK PORT ASSIGNMENT TO PORT CHANNEL ON FABRIC INTERCONNECT B ............................ 8

8.2 UPLINK PORT ASSIGNMENT TO PORT CHANNEL ON FABRIC INTERCONNECT A ............................ 8

9 SAN SWITCH CONNECTIVITY ................................................................................................................. 8

10 SERVICE PROFILES ............................................................................................................................. 8

10.1 SERVICE PROFILES USED .................................................................................................................... 8

10.2 SERVICE PROFILE TEMPLATE ............................................................................................................ 9

11 VNIC AND VHBA .................................................................................................................................. 9

12 POOLS ................................................................................................................................................. 9

Page 3 of 13 Confidential
12.1 UUID SUFFIX POOL ............................................................................................................................. 9

12.2 UUID POOL NAMING STANDARD USED .............................................................................................. 9

12.3 UUID TEMPLATE ............................................................................................................................... 10

12.4 MAC ADDRESS POOL ......................................................................................................................... 10

12.5 MAC POOL NAMING STANDARD ....................................................................................................... 10

12.6 MAC ADDRESS TEMPLATE ............................................................................................................... 11

13 WWN POOL ....................................................................................................................................... 11

13.1 WWNN POOL ..................................................................................................................................... 11

13.2 WWNN POOL TEMPLATE ................................................................................................................. 11

13.3 WWPN POOLS ................................................................................................................................... 11

13.4 WWPN POOL TEMPLATE .................................................................................................................. 12

14 VNIC AND VNIC TEMPLATES ............................................................................................................ 12

15 VHBA AND VHBA TEMPLATES ......................................................................................................... 12

16 VNIC & VHBA CREATION TEMPLATE ............................................................................................... 13

17 CHASSIS/FEX DISCOVERY POLICY ................................................................................................... 13

Page 4 of 13 Confidential
 1. INTRODUCTION
The Cisco Unified Computing System is an (x86) architecture next-generation data center platform
that unites compute, network, storage access, and virtualization into a cohesive system The Cisco
Unified Computing System (UCS) data center server platform composed of computing hardware,
virtualization support, switching fabric, and management software. The system integrates a low-
latency, lossless 10 Gigabit Ethernet unified network fabric with enterprise-class, x86-architecture
servers.
2. SYSTEM OVERVIEW
From a high-level perspective, the Cisco Unified Computing System consists of one or two Cisco UCS
Fabric Interconnects and one or more Cisco UCS 5100 Series Blade Server Chassis populated with
Cisco UCS B-Series Blade Servers. Cisco UCS Manager is embedded in the fabric interconnects, and it
supports all server chassis as a single, redundant management domain.
Each chassis requires at least one 10 Gigabit unified fabric connection to a Cisco Fabric Interconnect.
A typical configuration would have 2,4 or 8 unified fabric connections from each chassis to each of an
active-active pair of fabric interconnects.

2.1 UCS 5108 BLADE CHASSIS

Page 5 of 13 Confidential
 2.2 BLADE SERVERS:
The blade server chassis has flexible partitioning with removable dividers to handle two blade server
form factors:
• Half-width blade servers have access to power and two 10GBASE-KR connections, one to
each fabric extender slot.
• Full-width blade servers connect to power and two connections to each fabric extender.

3 UCS 62XX SERIES FABRIC INTERCONNECTS

Cisco UCS Fabric Interconnects is a family of low-latency, lossless 10 Gigabit Ethernet, Cisco DCE, and FCoE
interconnect switches that consolidate I/O at the system level. Based on the same switching technology as the
Cisco Nexus™ 5000 Series Switches, the Cisco UCS Fabric Interconnects provide the additional features and
management capabilities that make up the core of the Cisco Unified Computing System.

Deployed in active-active redundant pairs, the fabric interconnects provide uniform access to both
networks and storage Ethernet and FCoE downlinks and flexible 10 Gigabit Ethernet and 1/2/4/8-
Gbps Fibre Channel uplink.
4 UCS MANAGER
Cisco UCS Manager is the management system for all components in a Cisco UCS domain. Cisco UCS Manager
runs within the fabric interconnect.
Cisco UCS Manager abstracts server state information—including server identity, I/O configuration, MAC
addresses and World Wide Names, firmware revision, and network profiles—into a service profile. You can
apply the service profile to any server resource in the system
In a high availability environment with two fabric interconnects, you can run a separate instance of Cisco UCS
Manager on each fabric interconnect. The Cisco UCS Manager on the primary fabric interconnect acts as the
primary management instance, and the Cisco UCS Manager on the other fabric interconnect is the subordinate
management instance.

5 MAJOR COMPONENTS THAT COMPRISE UCS AT JK-SDC

Page 6 of 13 Confidential
 Component
1 x UCS 5108 chassis
2 x B200 M5 blade servers
2 x C220 Rack Servers
2 x C240 Rack Servers
2 x HX Servers
2 x UCS 6332 fabric interconnects

6 UCS CONFIGURATION INFORMATION

System Name xyz

Fabric Interconnect A IP
Fabric Interconnect B IP
Fabric Interconnect cluster IP
Default Gateway
Subnet Mask
DNS server
NTP Server
KVM/CIMC management IP
addresses for the blade
servers

7 PHYSICAL CONNECTIVITY

8 VLAN/SEGMENT USAGE
On each fabric interconnect, Multiple disjoint VLANs have been created. The name and number of the
VLANs are as follows:

The “default (1)” VLAN is the default VLAN and cannot be deleted and hence can be seen in above
figure.
On fabric interconnect A, a VSAN named VSAN_SW1_FIA_11 has been created. Associated FCoE VLAN
is 2011.
On fabric interconnect B, a VSAN named VSAN_SW2_FIB_12 has been created. Associated FCoE VLAN
is 2012.

Page 7 of 13 Confidential
 8.1 UPLINK PORT ASSIGNMENT TO PORT CHANNEL ON FABRIC
INTERCONNECT B

Uplink port assignment to Port Channel on Fabric Interconnect B is as below:

8.2 UPLINK PORT ASSIGNMENT TO PORT CHANNEL ON FABRIC

INTERCONNECT A
Uplink port assignment to Port Channel on Fabric Interconnect A is as below:
9 SAN SWITCH CONNECTIVITY

10 SERVICE PROFILES
Service profiles are the central concept of Cisco UCS. Each service profile serves a specific purpose:
ensuring that the associated server hardware has the configuration required to support the
applications it will host.
The service profile maintains configuration information about the server hardware, interfaces, fabric
connectivity, and server and network identity. This information is stored in a format that you can
manage through Cisco UCS Manager. All service profiles are centrally managed and stored in a
database on the fabric interconnect.
Every server must be associated with a service profile. At any given time, each server can be associated
with only one service profile. Similarly, each service profile can be associated with only one server at
a time.
10.1 SERVICE PROFILES USED

Service profile name Associated with the server OS

….. Continue!

Page 8 of 13 Confidential
 10.2 SERVICE PROFILE TEMPLATE
Considering the above recommendations while creating the service profile, we have created and
same been furnished below for reference.

11 VNIC AND VHBA

a. vNIC:
1. vNIC_FIA_DMZ_INT
2. vNIC_FIA_MxZ
3. vNIC_FIA_DB
4. vNIC_FIA_Mgmt

b. vHBA:
1. vHBA_FIA_C1_ SRV01
2. vHBA_FIB_C1_ SRV01

12 POOLS
In the full UCS deployment, one need to have identities defined with the logical service profile that
will then be applied to the Cisco UCS blade. A Logical Server is defined with identity (UUID,
MAC,WWNN and WWPN addresses).
12.1 UUID SUFFIX POOL
A UUID suffix pool is a collection of SMBIOS UUIDs that are available to be assigned to servers. The
first number of digits that constitute the prefix of the UUID are fixed. The remaining digits, the UUID
suffix, are variable. A UUID suffix pool ensures that these variable values are unique for each server
associated with a service profile which uses that particular pool to avoid conflicts.
12.2 UUID POOL NAMING STANDARD USED

(16 bit)
PREFIX - X X x x - x x x x x x x x x x x x

x x x
site id meaning domain meaning OS meaning
1 DC 1 V3 1 Windows
2 Linux
3 VMware

Page 9 of 13 Confidential
 selection bit (X X X) UUID pool name example

12.3 UUID TEMPLATE

Considered all 3 conditions as mentioned for UUID pool creation above for SITE, DOMAIN &
Operating System.

12.4 MAC ADDRESS POOL

A MAC pool is a collection of network identities, or MAC addresses, that are unique in their layer 2 environment
and are available to be assigned to vNICs on a server. If you use MAC pools in service profiles, you do not have
to manually configure the MAC addresses to be used by the server associated with the service profile.
To assign a MAC address to a server, you must include the MAC pool in a vNIC policy. The vNIC policy is then
included in the service profile assigned to that server.

12.5 MAC POOL NAMING STANDARD

MAC address is a 48-bit address. Out of 48, first 24 bits (00:25:B5) are reserved Cisco OUI and cannot be
changed. MCA V3 uses 3 bits (marked as red below) to identify site, fabric, and network segment (VLAN)
MAC address format: 00:25:B5:XX:XX:XX

X x X
site id meaning fabric meaning nw id meaning
1 DC A fabric A 1 MZ
1 DC B fabric B 2 DMZ_INT
1 DC A fabric A 2 MZ
1 DC B fabric B 2 DMZ_INT

selection bit (X X X) MAC pool name example

111 DC_FIA_MZ
121 DC_FIB_MZ
112 DC_FIA_DMZ_INT
122 DC_FIB_DMZ_INT

Page 10 of 13 Confidential
12.6 MAC ADDRESS TEMPLATE
Considering all above mentioned conditions, created the MAC address pool. Same thing copied below for
reference.
SITE ID, FABRIC ID & NETWORK ID

13 WWN POOL
A WWN pool is a collection of WWNs for use by the Fibre Channel vHBAs in a Cisco UCS domain. A separate pool
is created for the following:
WWNN assigned to the server
WWPN assigned to the vHBA
A WWN pool can include only WWNNs or WWPNs in the ranges from 20:00:00:00:00:00:00:00 to
20:FF:FF:FF:FF:FF:FF:FF or from 50:00:00:00:00:00:00:00 to 5F:FF:FF:FF:FF:FF:FF:FF. All other WWN ranges
are reserved. To ensure the uniqueness of the Cisco UCS WWNNs and WWPNs in the SAN fabric, it is
recommended to you use the following WWN prefix for all blocks in a pool: 20:00:00:25:B5:XX:XX:XX

13.1 WWNN POOL

A WWN pool can include only WWNNs or WWPNs in the ranges from 20:00:00:00:00:00:00:00 to
20:FF:FF:FF:FF:FF:FF:FF or from 50:00:00:00:00:00:00:00 to 5F:FF:FF:FF:FF:FF:FF:FF. All other WWN ranges
are reserved. To ensure the uniqueness of the Cisco UCS WWNNs and WWPNs in the SAN fabric, we recommend
that you use the following WWN prefix for all blocks in a pool: 20:00:00:25:B5:XX:XX:XX
If you use WWN pools in service profiles, you do not have to manually configure the WWNs that will be used by
the server associated with the service profile. In a system that implements multi-tenancy, you can use a WWN
pool to control the WWNs used by each organization.

13.2 WWNN POOL TEMPLATE

Considering above mentioned recommendations, created the WWNN pool and same been copied here for
reference.

13.3 WWPN POOLS

A WWPN pool is a WWN pool that contains only WW port names. If you include a pool of WWPNs in a
service profile, the port on each vHBA of the associated server is assigned a WWPN from that pool.
WWPN Pool naming standard used
WWPN format: 20:00:00:25:B5:XX:XX:XX

x X x
fabric id meaning chassis Meaning server meaning
A fabric A 1 chassis 1 1 server 1
B fabric B 2 server 2

Page 11 of 13 Confidential
 13.4 WWPN POOL TEMPLATE
Considering the above recommendations for WWPN pool creations, created the same and copied here
for reference. It consists FABRIC ID, SERVER ID & CHASIS ID.

14 VNIC AND VNIC TEMPLATES

A vNIC is a virtualized network interface that is configured on a physical network adapter and appears to be a
physical NIC to the operating system of the server.
A vNIC communicates over Ethernet and handles LAN traffic. At a minimum, each vNIC must be configured with
a name and with fabric and network connectivity.
The vNIC template defines how a vNIC on a server connects to the LAN. This policy is also referred to as a vNIC
LAN connectivity policy.
This policy requires that one or more of the following resources already exist in the system:
Named VLAN
MAC pool
QoS policy
LAN pin group
Statistics threshold policy

To provision automatic vNIC creation, following vNIC templates have been created and are used in service
profiles:

15 VHBA AND VHBA TEMPLATES

A vHBA is a virtualized host bus adapter that is configured on a physical network adapter and appears to be a
physical HBA to the operating system of the server. A vHBA communicates over FCoE and handles SAN traffic.
At a minimum, each vHBA must be configured with a name and fabric connectivity.

This template is a policy that defines how a vHBA on a server connects to the SAN. It is also referred to as a
vHBA SAN connectivity template.
You need to include this policy in a service profile for it to take effect.
This policy requires that one or more of the following resources already exist in the system:
Named VSAN
WWNN pool or WWPN pool
SAN pin group
Statistics threshold policy
In order to provision automatic vHBA creation, following vHBA templates have been created and are used in
service profiles:

Page 12 of 13 Confidential
16 VNIC & VHBA CREATION TEMPLATE

17 CHASSIS/FEX DISCOVERY POLICY

The chassis/FEX discovery policy determines how the system reacts when you add a new chassis or FEX. Cisco
UCS Manager uses the settings in the chassis/FEX discovery policy to determine the minimum threshold for the
number of links between the chassis or FEX and the fabric interconnect and whether to group links from the IOM
to the fabric interconnect in a fabric port channel.
Cisco UCS Manager cannot discover any chassis that is wired for fewer links than are configured in the
chassis/FEX discovery policy. For example, if the chassis/FEX discovery policy is configured for 2 links, Cisco
UCS Manager cannot discover any chassis that is wired for 1 link.
The Chassis/FEX Discovery policy for the UCS system :

Page 13 of 13 Confidential