UCS and ACI Integration

Integrating Cisco UCS with Cisco
ACI
Marian Klas, mklas@cisco.com

Systems Engineer – Data Center
February 2015
Agenda:
§ Connecting workloads to ACI

§ Bare Metal
§ Hypervisors
§ UCS & APIC Integration and Orchestration
§ Converged Stack Examples
© 2015 Cisco and/or its affiliates. All rights reserved. 2

Connecting workloads to ACI:
Bare Metal Servers

Connecting Physical Workloads to a Port
Infrastructure configuration (Access Policy)
• Switch 101
Switch Profile • Switch 102
• Etc…
• Interface 1/1
Interface Profile • Interface 1/2
• Etc…
• Aggregates port configurations

Policy Group = port
configuration • It can be of type Interface, Port-
channel, vPC etc…
4
“Enabling” VLANs on a set of ports
What is it? It’s just a way to tell the fabric which VLAN ranges
are expected where, it’s useful for VLAN reuse
Attach Entity • Aggregates Domains and has a

Profile reference with the policy-group
Physical • Can be made of multiple VLAN pools

Domain
VLAN Pool • Defines a range of VLANs
5
Mapping between Tenant View and Infrastructure view
Tenant View Infrastructure View
maps to port, VLAN

Port (i.e. port- Attach Entity
Policy Group = port
EPG profile, switch
profile)
configuration Profile
VLAN must be part of AEP

How to Connect Servers
Tenant leaf portchannel
virtual machine
virtual machine
EPG, bridge domain, router
Mapping an EPG to a Server Port

Endpoints are discovered
Under Operational Client Endpoints

EPG mapped to multiple VLANs on different leafs
Spines
Leafs Border Leafs
vlan30 vlan10 vlan20 vlan40

ACI Fabric – Endpoint Connectivity
Rack Mount Servers Policy Enforcement
9300 9300
9300 9300
10G
1/10G
40G
N2K N2K
No Host vPC
9396PX/TX or 93128TX 9300 Leaf w/ FEX

Leaf
• Flexible teaming options including:
• Direct Connect, FEX Supported – Active/Active w/ LACP;
– 9300 up to 6 x FEX (as of Jan 2015) – Active/Active with AVS and FEX
– Enhanced vPC and vPC w/ FEX not Supported (as of Jan
2015)
© 2015 Cisco and/or its affiliates. All rights reserved.
– Active/Standby 13
13
FEX Topology Support Roadmap
Active/Standby Straight Through
vPC (Dual Homed) EvPC
Teaming (Single Homed)
Nexus 9300
Standalone
6.1(2)I2(3) 6.1(2)I2(3) Future Future
Nexus 9300
ACI Leaf
11.0(1d) - Shipping 11.1(x) - 1HCY15 Future Future

FEX Support
Nexus 9300 FEX Support ACI FEX Support
• 6.1(2)I2(3) • 11.0(1d) - Shipping
• N2224TP, N2248TP, N2248TP-E, • N2K-C2248PQ, N2K-C2248TP-E,
N2232TM, N2232PP, B22HP N2K-C2248TP-1GE, N2K-
C2232PP-10GE, N2K-C2232TM-E
• 6.1(2)I3(1)
• 11.1(x) - Q1CY15
• B22-Dell, 2232TM-E, 2248PQ FEX
• B22HP, B22-DEL, B22-IBM
• Q1 CY15
• Q2CY15
• B22-IBM, B22-Fujitsu, 2348UPQ
• 2348UPQ

ACI Fabric – Endpoint Connectivity
3rd Party Blade Servers & Ethernet northbound connectivity
Policy Enforcement
L2 Switching
Virtual Switching
ACI Fabric
N9K N9K N9K N9K N9K N9K N9K N9K
N2K N2K
PT PT B22 B22 HPVC HPVC SW SW
No Host vPC No Host vPC

Blade Switches from:
B22-HP Flex-10 • Cisco
B22-Dell FlexFabric • HP
Passthrough B22-IBM • Force-10
• IBM/BNT
• Blade Servers – HP, IBM, Dell. (Q1CY15 – MR2 release)
• 3rd Party Switches – STP Interoperability
• Leaf Port Type – Network/Edge
– Edge Port – BPDU Guard
– Network Port – BPDU Pass-Through
17
Connecting workloads to ACI:
Hypervisor Integration

Hypervisors Integration
• General concepts
• Integration with Vmware

• Cisco Application Virtual Switch (AVS)
• VMware DVS
• Integration with Hyper-V
• Integration with KVM / Openstack
19
Hypervisor Interaction with ACI
• Two modes of Operation
Non-Integrated Mode Integrated Mode
VLAN 10 VLAN 10 VXLAN 10000 APP WEB DB DB
• ACI Fabric as an IP-Ethernet • ACI Fabric as a Policy Authority

Transport
• Encapsulations Normalized and
• Encapsulations manually allocated dynamically provisioned
• Separate Policy domains for Physical • Integrated Policy domains across
and Virtual Physical and Virtual
20
Hypervisor Integration with ACI
• Control Channel - VMM Domains
§ Relationship is formed between APIC

and Virtual Machine Manager (VMM)
§ Multiple VMMs likely on a single ACI

Fabric
§ Each VMM and associated Virtual hosts

are grouped within APIC
vCenter DVS vCenter AVS SCVMM § Called VMM Domain
§ There is 1:1 relationship between a
Virtual Switch and VMM Domain
VMM Domain 1 VMM Domain 2 VMM Domain 3
21
§ ACI Fabric implements policy on Virtual
APIC Networks by mapping Endpoints to
EPGs
§ Endpoints in a Virtualized environment
are represented as the vNICs
§ VMM applies network configuration by
placement of vNICs into:
Application Network Profile § Port Groups (VMWare),
EPG EPG EPG
F/W WEB L/B APP DB § VM Networks (Hyper-V)
§ Networks (OpenStack)
WEB PORT GROUP APP PORT GROUP DB PORT GROUP § EPGs are exposed to the VMM as a 1:1
mapping to Port Groups, VM Networks
VM VM VM
or OpenStack Networking.
22
• Endpoint Discovery
§ Virtual Endpoints are discovered for

reachability & policy purposes via 2 APIC
methods:
§ Control Plane Learning:
- Out-of-Band Handshake: vCenter APIs
- Inband Handshake: OpFlex-enabled
Host (AVS, Hyper-V, etc.) Control
§ Data Path Learning: Distributed (vCenter API)
switch learning
§ LLDP/CDP used to resolve Virtual VMM
host ID to attached port on leaf node Control Data Path Data Path
(non-OpFlex Hosts) (OpFlex)
DVS Host
OpFlex Host
26

• VMware DVS
27
VMWare Integration
• Three Different Options
Distributed Virtual Switch Application Virtual Switch

vCenter + vShield
(DVS) (AVS)
• Encapsulations: VLAN • Encapsulations: VLAN, • Encapsulations: VLAN,

• Installation: Native VXLAN VXLAN
• VM discovery: LLDP/ • Installation: Native • Installation: VIB through

CDP • VM discovery: LLDP/ VUM or Console
• Software/Licenses: CDP • VM discovery: OpFlex

vCenter with • Software/Licenses: • Software/Licenses:
EnterprisePlus License vCenter with vCenter with
EnterprisePlus License, EnterprisePlus License
vShield Manager with
vShield License
28
ACI Hypervisor Integration – VMware DVS/vShield
Application Network Profile
5 EPG EPG
EPG DB
WEB APP
APIC L/B
F/W
Create Application Policy
APIC Admin
ACI
9
Fabric
Push Policy
1
6
Cisco APIC and VMware Learn location of ESX
4
vCenter Initial Automatically Map Host through LLDP
Handshake EPG To Port Groups
VIRTUAL DISTRIBUTED SWITCH

2 Create VDS
WEB PORT GROUP APP PORT GROUP DB PORT GROUP
Create Port
vCenter 7
Groups
Server / vShield
Web App DB Web Web DB
8 Attach Hypervisor
VI/Server Admin Instantiate VMs, to VDS
Assign to Port Groups 3
HYPERVISOR HYPERVISOR
29
ACI Hypervisor Integration – VMware DVS
Name of VMM Domain

Type of vSwitch (DVS or AVS)
Associated Attachable Entity Profile (AEP)
VLAN Pool
vCenter Administrator Credentials
vCenter server information
30
ACI Hypervisor Integration – VMware DVS
31
ACI Hypervisor Integration – AVS
5 EPG EPG EPG
WEB APP DB
APIC L/B
F/W
Create Application Policy
APIC Admin
ACI
9
Fabric
Push Policy
1
6
Cisco APIC and VMware Learn location of ESX
4
vCenter Initial Automatically Map Host through OpFlex
Handshake EPG To Port Groups
OpFlex Agent OpFlex Agent
Create AVS Application Virtual Switch (AVS)

2
VDS
WEB PORT GROUP APP PORT GROUP DB PORT GROUP
Create Port
vCenter 7
Groups
Server
Web App DB Web Web DB
8 Attach Hypervisor
VI/Server Admin Instantiate VMs, to VDS
Assign to Port Groups 3
HYPERVISOR HYPERVISOR
33
Extending ACI to Existing Virtual & Physical Network
Phase 1: Layer 2 Existing
Network/Local Switching
§ AVS supports OpFlex to integrate
with APIC
§ Supports a Full multi-hop Layer 2
Network between Nexus 9k and
AVS: Investment Protection
OpFlex
§ VMware DVS can only support a
OpFlex
single L2 switch between N9k and
OpFlex
DVS AVS
§ LLDP/CDP and NOT OpFlex
Integration AVS
§ Layer 2 network is required to

support OpFlex bootstrapping in AVS Layer 2
this phase
Network


• VMware DVS
36
Microsoft Interaction with ACI
• Two modes of Operation
Integration with SCVMM Integration with Azure Pack
APIC APIC +
• Policy Management: Through APIC • Superset of SCVMM

• Software / License: Windows Server with • Policy Management: Through APIC or
HyperV, SCVMM through Azure Pack
• VM Discovery: OpFlex • Software / License: Windows Server with
HyperV, SCVMM, Azure Pack (free)
• Encapsulations: VLAN, NVGRE (Future)
• VM Discovery: OpFlex
• Plugin Installation: Manual
• Encapsulations: VLAN, NVGRE (Future)
• Plugin Installation: Integrated
37
Microsoft Azure Pack Integration
§ Integration with Microsoft requires: Customer

Service Provider
- Windows Server 2012

Service Web Sites
- Systems Center 2012 R2 with SPF Plans
Provider
Apps Consumer
Users
Portal
Database Self-Service
- Windows Azure Pack VMs
ACI
Portal
§ Azure Pack provides single pane of

glass for Definition, creation,
management of their cloud service
§ Divided into Provider (Admin) portal Web
Sites
VMs SQL
Service
Bus …
and Consumer Self-Service (Tenant)
portal R2 w/ Service Provider
Foundation
§ Cisco ACI Service Plugin enables
management of Network Infrastructure
through APIC REST API
39
ACI Azure Pack Integration
1
APIC
APIC Admin
(Basic Infrastructure)
7 ACI
Fabric
Pull Policy on leaf

where EP attaches
3 2
Get VLANs allocated Push Network

for each EPG Profiles to APIC
6
5
Create VM Networks Indicate EP Attach to attached leaf
Create Application 4 when VM starts
Instantiate VMs
Policy
1 4
APIC Plugin SCVMM Plugin OpFlex Agent OpFlex Agent OpFlex Agent
HYPERVISOR HYPERVISOR HYPERVISOR
Azure Pack \ SPF

Azure Pack Tenant
Web App Web App DB Web Web DB
41

• VMware DVS
42
ACI OpenStack Integration – Phase 1
3
APIC Create Application Policy
APIC Admin
(Performs Steps 3)
ACI
5
Fabric
Push Policy
Automatically Push
Network Profiles to
APIC
Create Network, Subnet,
Security Groups, Policy
NETWORK ROUTING SECURITY
1
OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH
NEUTRON NOVA
4 Web App Web App DB Web Web DB
OpenStack Tenant
(Performs Steps 1,4) Instantiate VMs
© 2015 Cisco and/or its affiliates. All rights reserved. HYPERVISOR HYPERVISOR HYPERVISOR 46
46
ACI OpenStack Integration – Phase 2 (Group-based policy)
Create Application Network
Profile Application Network Profile
EPG EPG EPG
F/W WEB APP DB
1 L/B
L/B
NEUTRON NOVA
4 Web App Web App DB Web Web DB
OpenStack Tenant
(Performs step 1,4) Instantiate VMs
HYPERVISOR HYPERVISOR HYPERVISOR
2 Automatically Push
Network Profiles to
APIC

3 EPG EPG EPG
F/W WEB APP DB
APIC Create Application Policy L/B L/B
ACI Admin
(manages physical
network, monitors tenant
state) ACI
5
Fabric
Push Policy
UCS & APIC Integration

ACI Management Overview with UCS End-Host Mode
UCS Director
SAN A SAN B
UCSM /
UCS Central
Ethernet and
Ethernet FCOE
FC FCOE 51
ACI Management Overview with UCS End-Host Mode
UCS Director
SAN A SAN B
UCS Stand-alone
EPG
UCS Director
SP Binding
UCSM/ - VLANs
UCS Central
UCSM /
UCS Central
Ethernet and
Ethernet FCOE
FC FCOE 52
UCSD Application Provisioning Lifecycle
Create VMs
Assign to EPGs
Assign Baremetal Servers to EPGs
UCS Director A B C
VM VM VM
Virtualization Baremetal
Application
Catalog Web
Create SharePoint
VM VM
App Policy in APIC
L4-7
VM VM Compute
App
VM VM Fabric
L4-7
Create Additional Storage
for DB Tier
DB Server Load
Balancer
L4-7 Services
Storage
Manager
© 2015 Cisco and/or its affiliates. All rights reserved.

Application Fully Instantiated Storage
57
Common Policy Based Infrastructure Programmability
Flexibility, Performance, and Visibility
§ Consistent Virtualized and Physical
Server Deployment Models
§ Multi-Phase Approach
§ All phases show a Cisco together value add
§ First 2 on existing UCS deployments
§ Later one on 3rd Generation UCS FI
§ Phase 1 (2014): UCS Director to

orchestrate workloads over interlinked
UCS and ACI
§ Phase 2 (2015): UCS and ACI sharing
policy and state for a better together
story Automated Policy and State Integrated
Configuration via Exchange via Management as
UCS Director Software Agent Native ACI Leaf
§ Phase 3 (Planning): 3rd Gen UCS UCS FI’s
Fabric Interconnect is a leaf with
expanded policy and state interaction
Phases and Tradeoffs – UCSM Managed Servers
Feature Phase 1 Phase 2a Phase 2b Phase 3
UCS Fabric 6100/6200 6100/6200 6300 6300

Interconnects
UCS to ACI Leaf vPC N:1 (10GE) N:1 (10GE) N:1 (40GE) 1:1 (Can be a Leaf)
Pair Ratio
Port Types/Speeds 10GE SFP, 8GFC SFP 10GE SFP, 8GFC SFP 40GE QSFP, 16GFC 40GE SFP, 16GFC SFP
SFP
UCS Manager Location Within FI Within FI Within FI Decoupled and
containerized
Policy Integration Agent UCS Director Over the Top Bridge OpFlex Control Path OpFlex Control Path
between DME’s
UCS Physical Connected to ACI Leaf Connected to ACI Leaf Connected to ACI Leaf Is ACI Leaf – Connected
Attachment to ACI Spine
Health Scoring To ACI Leaf Intra UCS fabric data Inside OpFlex Path Direct from UCS FI ASIC
feed to Observer and OS
End Point Group Construction of EPG to Auto-Population of EPG Auto-Population of EPG EPG assignment and
Mechanisms VLAN within UCSD to VLAN data to VLAN data encapsulation per UCS
server interface
Converged stack examples

ACI-ready Vblock and FlexPOD
http://newsroom.cisco.com/press-release-content?
type=webcontent&articleId=1421361
63
FlexPod with ACI - Cisco and Partner Technologies
Key Cisco and Key Components of FlexPod

Partner Technologies Application ACI Fabric / Nexus 9000 with ACI design
Policy spine / leaf
Infrastructure § Nexus 9500/9300 Spine & Leaf
Controller
Unified Compute System Switches
Blade and Rack mount
Servers § 3 node Cisco Application Policy
APIC Infrastructure Controller (APIC)
Nexus 9000 Top of Rack and
Modular switches § UCS Manager 2.2 release
§ vSphere 5.5 update 2
NetApp Clustered Data
ONTAP utilizing storage virtual § L4-L7 Services, ASA Firewall
machines and F5 Load balancer
VMware Hypervisor and § NetApp FAS 8000 and

Hypervisor CDOT 8.2.1
vCenter management Managers
§ OnCommand System Manager

Firewall and Load Balancer Load
Services Balancer
§ NetApp Snap Manager
NetApp Clustered Integrated Cisco Unified

Hypervisor L4-L7 Computing
ONTAP
Services System
FlexPod Data Center with ACI
§ FlexPod Data Center pre-validated

Integration with ACI
§ Configuration management using GUI in
the current release – UCS Director to be
incorporated in upcoming designs
§ Application Validation
§ Microsoft Exchange 2013
§ Microsoft SharePoint 2013
§ Microsoft SQL Server 2012 SP1
§ Additional Applications to be validated in

future

Thank you.

UCS and ACI Integration

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

UCS and ACI Integration

Uploaded by

Copyright:

Available Formats

Integrating Cisco UCS with Cisco

Marian Klas, mklas@cisco.com

§ Connecting workloads to ACI

© 2015 Cisco and/or its affiliates. All rights reserved. 2

© 2015 Cisco and/or its affiliates. All rights reserved. 3

• Aggregates port configurations

© 2015 Cisco and/or its affiliates. All rights reserved. 4

Attach Entity • Aggregates Domains and has a

Physical • Can be made of multiple VLAN pools

VLAN Pool • Defines a range of VLANs

© 2015 Cisco and/or its affiliates. All rights reserved. 5

Tenant View Infrastructure View

maps to port, VLAN

VLAN must be part of AEP

© 2015 Cisco and/or its affiliates. All rights reserved. 6

Tenant leaf portchannel

© 2015 Cisco and/or its affiliates. All rights reserved. 9

© 2015 Cisco and/or its affiliates. All rights reserved. 11

Leafs Border Leafs

vlan30 vlan10 vlan20 vlan40

© 2015 Cisco and/or its affiliates. All rights reserved. 12

9396PX/TX or 93128TX 9300 Leaf w/ FEX

6.1(2)I2(3) 6.1(2)I2(3) Future Future

11.0(1d) - Shipping 11.1(x) - 1HCY15 Future Future

© 2015 Cisco and/or its affiliates. All rights reserved. 15

PT PT B22 B22 HPVC HPVC SW SW

No Host vPC No Host vPC

© 2015 Cisco and/or its affiliates. All rights reserved. 18

• Integration with Vmware

• Integration with KVM / Openstack

© 2015 Cisco and/or its affiliates. All rights reserved. 19

Non-Integrated Mode Integrated Mode

VLAN 10 VLAN 10 VXLAN 10000 APP WEB DB DB

• ACI Fabric as an IP-Ethernet • ACI Fabric as a Policy Authority

§ Relationship is formed between APIC

§ Multiple VMMs likely on a single ACI

§ Each VMM and associated Virtual hosts

© 2015 Cisco and/or its affiliates. All rights reserved. 21

§ Virtual Endpoints are discovered for

• Integration with Vmware

• Integration with KVM / Openstack

© 2015 Cisco and/or its affiliates. All rights reserved. 27

Distributed Virtual Switch Application Virtual Switch

• Encapsulations: VLAN • Encapsulations: VLAN, • Encapsulations: VLAN,

• VM discovery: LLDP/ • Installation: Native • Installation: VIB through

• Software/Licenses: CDP • VM discovery: OpFlex

VIRTUAL DISTRIBUTED SWITCH

Name of VMM Domain

vCenter Administrator Credentials

vCenter server information

© 2015 Cisco and/or its affiliates. All rights reserved. 30

© 2015 Cisco and/or its affiliates. All rights reserved. 31

OpFlex Agent OpFlex Agent

Create AVS Application Virtual Switch (AVS)

§ Layer 2 network is required to

© 2015 Cisco and/or its affiliates. All rights reserved. 34

• Integration with Vmware

• Integration with KVM / Openstack

© 2015 Cisco and/or its affiliates. All rights reserved. 36

Integration with SCVMM Integration with Azure Pack

• Policy Management: Through APIC • Superset of SCVMM

© 2015 Cisco and/or its affiliates. All rights reserved. 37