ACI Anywhere

ACI Fabric Design
Choose the type of ACI fabric you want to implement

• ACI Mini for a maximum of 2 spines and
4 leafs:
• Starting with ACI 4.0.1 you can deploy a Mini ACI fabric
with virtual APICs (vAPIC) for up to two spines and four
leafs
• Supports small ACI fabric deployments: Up to 200 edge
ports.
• Migrate to a full ACI fabric by replacing virtual APICs with
Physical APICs
• ACI 2-tier fabric:

• This is the classic ACI topology
• ACI 3-tier fabric: Physical APIC

Virtual APIC Virtual APIC
• This type of topology has been introduced in ACI 4.1 to
meet the cabling requirements of many datacenters
More about ACI Mini at this link:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/Cisco-Mini-ACI-Fabric-and-Virtual-APICs.html
#CiscoLiveLA TECACI-2009 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
The classic ACI spine-leaf architecture
ACI 2-tiers architecture
• Define which scale profile

applies to which leaf
• Define which uplinks need to be vPC vPC
modified to be downlinks
• Define which leafs are vPC
pairs
• FEX optional
Addressing Cabling Requirements
• The 3-tiers topology simplifies the
integration of ACI with the Existing Cable
Plan
• The scalability is the same as a 2-tier Spines

fabric
• First generation leafs can be part of the

fabric but not for connecting Tier-2 leafs Tier-1 Leaf
• Spines:
• Any -EX/FX, N9364C, N9332C
Tier-2 Leaf
• Tier-1 Leafs:
• Any –EX, –FX & -FX2 ToR
• (Exception: N93180LC-EX)
• Tier-2 Leafs:
• Any –EX, –FX & -FX2 ToR
• Please refer to the documentation for the

details of how many ports can be used for
fabric on each model
Connectivity requirement to Tier-1 Leafs
• All ports of Tier-1 Leafs can be

converted to be fabric ports using the
port profile feature
• Tier-2 Leaf fabric ports connect to Tier-

1 Leafs' fabric ports
• APIC, L3out, EPGs can be connected to

Tier-1/Tier-2 leafs
Tier-2 Leaf can connect to
multiple Tier-1 Leafs: ECMP
If APIC is connected to a tier-2 leaf:

one default fabric port of the Tier-2 leaf must be connected to a default fabric port of a tier-1 leaf.
If no APIC is connected to a tier-2 leaf:

one default fabric port of the Tier-2 leaf must be connected to any port of a tier-1 leaf.
https://www.cisco.com/c/en/us/solutions/data-center-virtualization/application-centric-infrastructure/white-paper-c11-742214.html
ACI leafs can also provide Fibre Channel and FCoE Connectivity
ACI leaf as an FCoE/FC NPV switch
• ACI FC Features:
• San Port Channel FC
Storage
• Trunking FC NP link – supporting multiple VSANs
• San Boot N5K
NPIV
core N6K
switch N7K
MDS
TRUNKING FC NP-port
• More information at:
• https://www.cisco.com/c/en/us/td/docs/switches/datacenter/
aci/apic/sw/4-x/L2-configuration/Cisco-APIC-Layer-2-
Configuration-Guide-411/Cisco-APIC-Layer-2-
Configuration-Guide-411_chapter_01000.html FC HOST PORT
VFC Port
• https://www.cisco.com/c/en/us/td/docs/switches/datacent
er/aci/apic/sw/4-x/L2-configuration/Cisco-APIC-Layer-2- N-Port VN Port
Configuration-Guide-411/Cisco-APIC-Layer-2-
Configuration-Guide-411_chapter_01001.html
FC Initiator FCoE Initiator
Border Leaf Choice: VRF-Lite or GOLF
VRF-Lite with L3out Border Leaf (s) GOLF (no border leaf)
RR RR MP-BGP EVPN with Nexus 7k or ASR9k
Border Leaf (s)

VXLAN
RR RR
BD subnets for VRF1 BD subnets for VRF2
More about the L3out later
#CiscoLiveLA TECACI-2009 20
ACI Anywhere,
Extending the ACI
Fabric
Agenda
§ ACI Anywhere, Extending the ACI Fabric
§ Overall Design Principles (AZs and Regions)
§ Mapping use cases to the proper solutions
Ø Active/Active DC à Multi-Pod
Ø Disaster Recovery à Multi-Site
Ø Migration/Coexistence with Legacy DC Networks and ‘Disaggregated DCs’
Model à Physical Remote Leaf
Ø Baremetal Cloud Integration à Virtual Pod (vPod)
§ Extending ACI to the Cloud
§ Connecting the users to the Multi-Cloud DC
Ø ACI and SDA Integration
Ø ACI and SDWAN Integration
Systems Availability
Best of Breed prior to 2014
• Distinct Network Domains for availability
§ Extension of Layer 2 works but complicates change and fault isolation
§ Sizing of each domain is a balance between need, risk and cost
Layer 2 Extension
VPLS/OTV/…
Layer 3
Layer 2
Network Fault/Change/Reachability Domain Network Fault/Change/Reachability Domain
Does ACI Change Anything?
YES
Reachability is now Decoupled from Fault and
Change
Reachability is Decoupled from Topology as well

Topology Flexibility
ACI Leaf Can Be Independent of the “Leaf-Spine” Topology
Disaster Avoidance and Disaster Recovery
Key Differences
Main/Active DCs
“Disaster Avoidance (DA)” “Disaster Recovery (DR)” Disaster-Recovery /
Planned / Unplanned Service Loss of “Regional data centers”
Business Continuity Hybrid-cloud
Continuity within the Metro without leads to recovery in a Remote data
with no interruption interruption of Services center •Long Distances
•Shorter Distances
(including a single data center loss) •Beyond app latency
•Synchronous Replication •Asynchronous Rep.
•Move Apps
•Low latency
•Distribute Apps • not distribute
•Cold Migration:
•Hot Live Migration
•Active/Active DCs • Stateless Services
•Integration of Stateful •Public/Hybrid Cloud
Dev •Subnet Extension
•Private Cloud •RTO > hours/days
•LAN extension •RPO > several secs to min
•RTO/RPO~=0
DC-1 DC-2 DC-3
Active-Active
Metro Area Pervasive Data Protection

+ Infrastructure Rebalance
Yesterday
Data Center Interconnect Solutions
A Tale from the Past
Over dark fiber or protected D-WDM
Ø vPC double-sided (caution with DWDM SLA) Metro style
Ethernet
§ Dual site interconnection
Ø OTV or VXLAN EVPN
§ Dual/Multiple sites interconnection
MPLS Transport
Ø MPLS-EVPN
§ SP, Point to Multipoint SP style
MPLS
Ø PBB-EVPN
§ Large scale & Multi-tenants, Point to Multipoint
IP Transport
Ø OTV
§ Interconnect Traditional-based DC Network IP style
IP Ø VXLAN EVPN
§ interconnect VXLAN-based Fabric
§ Layer 2 Ext. only and/or Layer 3 Ext. (multitenancy)
Ø LISP
§ For Subnet extension and Path Optimization
Today
Data Center Interconnect Solutions
ACI Simplifies the Deployment of DCI
• Common Control/Data Plane options used across different architectures
• Consistent security policies end-to-end
ACI Multi-Pod Fabric ACI Multi-Site
IPN IP
Pod ‘A’ Pod ‘n’ Fabric ‘A’ Fabric ‘n’
MP-BGP - EVPN MP-BGP - EVPN
… …
APIC Cluster
ACI Physical Remote Leaf ACI Virtual Remote Leaf (vPod)
Multi-Pod or Multi-Site?
That is the question…
And the answer is…
EASY
BOTH!
Framework for Multi-Cloud High Availability Design
Regions and Availability Zones
OpenStack
• Regions - Each Region has its own full OpenStack
deployment, including its own API endpoints, networks
and compute resources
• Availability Zones - Inside a Region, compute nodes can
be logically grouped into Availability Zones, when launching
new VM instance, we can specify AZ or even a specific
node in a AZ to run the VM instance
• Regions – Separate large geographical areas, each Amazon Web Services

composed of multiple, isolated locations known as
Availability Zones
• Availability Zones - Distinct locations within a region
that are engineered to be isolated from failures in other
Availability Zones and provide inexpensive, low latency
network connectivity to other Availability Zones in the
same region
Typical Requirement
Creation of Two Independent Fabrics/AZs
Fabric ‘A’ (AZ 1)
Fabric ‘B’ (AZ 2)
Application
workloads deployed
across availability
zones
Typical Requirement
Creation of Two Independent Fabrics/AZs
Multi-Pod Fabric ‘A’ (AZ 1)
‘Classic’ Active/Active
Pod ‘1.A’ Pod ‘2.A’
ACI Multi-Site
Multi-Pod Fabric ‘B’ (AZ 2)
‘Classic’ Active/Active
Pod ‘1.B’Application Pod ‘2.B’

workloads deployed
across availability
zones
Agenda
Active/Active DC Deployment
Metro Virtual Data Centre
“Disaster Avoidance (DA)” “Disaster Recovery (DR)”
§ High-availability application and data solution architecture which

leverages a dual data centre physical infrastructure (tightly DC-1
Active-Active
DC-2 DC-3
coupled DC for short distances)

§ Management and interaction of applications in a paired data

centre environment
§ Disaster Avoidance and Prevention by pro-actively migrates
seamlessly Virtual Machines with no interruption (vMotion for
example)
§ Active-active capability and workload rotation to accelerate
incident response time and increase confidence
§ Deployment of an ESXi Metro Cluster with vSphere HA, Fault
Tolerance (FT), DRS
§ Service Nodes (FW, SLB) clustered across DCs (Active/Standby,
Active/Active)
For More Information on
ACI Multi-Pod ACI Multi-Pod:
The Ideal Architecture for Active/Active DC Deployments BRKACI-2003
VXLAN
Inter-Pod Network
Pod ‘A’ Pod ‘n’
MP-BGP - EVPN
…
50 msec RTT
APIC Cluster
IS-IS, COOP, MP-BGP IS-IS, COOP, MP-BGP
Availability Zone
§ Multiple ACI Pods connected by an IP Inter-Pod L3 § Forwarding control plane (IS-IS, COOP) fault
network, each Pod consists of leaf and spine nodes isolation
§ Managed by a single APIC Cluster § Data Plane VXLAN encapsulation between Pods
§ Single Management and Policy Domain § End-to-end policy enforcement
125
Single Availability Zone with Tenant Isolation
Isolation for ‘Virtual Network Zone and Application’ Changes
Inter-Pod Network
ACI Multi-Pod
Fabric
APIC Cluster
Tenant ‘Prod’ Configuration/Change Domain

Tenant ‘Dev’ Configuration/Change Domain
§ The ACI ‘Tenant’ construct provide a domain of application and associated virtual network
policy change
§ Domain of operational change for an application (e.g. production vs. test)
ACI Multi-Pod
Common Use Cases
§ Need to scale up a single ACI fabric above
Pod
200 leaf nodes supported in a single Pod
Inter-Pod
§ Handling 3-tiers physical cabling layout (for Leaf Nodes Network
example traditional N7K/N5K/N2K
deployments)
Spine Nodes
§ True Active/Active DC deployments

Pod 1 Pod 2
Single VMM domain across DCs (stretched ESXi
Metro Cluster, vSphere HA/FT, DRS initiated
workload mobility,…)
Deployment of Active/Standby or Active/Active
clustered network services (FWs, SLBs) across DCs APIC Cluster
DB Web/App Web/App
Application clustering (L2 BUM extension across
Pods)
ACI Multi-Pod
APIC Cluster Deployment
APIC – Distributed Multi-Active Data Base
One copy is ‘active’ for every

The Data Base is replicated specific portion of the Data
across APIC nodes Base
(3 copies)
Shard 1 Shard 1 Shard 1

APIC APIC APIC
Shard 2 Shard 3 Shard 2 Shard 3 Shard 2 Shard 3
• All Services in ACI run against their own portions of a Database

• Services and Database Processes are active on all nodes (not active/standby)
• The Data Base is distributed as active + 2 backup instances (shards) for every attribute
APIC Cluster Deployment Considerations
Multi-Pod – 2 Pods Scenario
Pod 1
X Pod 2 Pod 1
X Pod 2
Up to 50 msec Up to 50 msec
X X
APIC APIC APIC APIC
X X X
APIC APIC APIC APIC APIC
Read/Write Read Only

§ Pod isolation scenario: same considerations as with
§ Pod isolation scenario: changes still possible single Pod (different behaviour across shards)
on APIC nodes in Pod1 but not in Pod2 § Pod hard failure scenario: may cause the loss of
§ Pod hard failure scenario: recommendation is information for the shards replicated across APIC
to activate a standby node to make the nodes in the failed Pod
cluster fully functional again Possible to restore the whole fabric state to the latest taken
configuration snapshot (‘ID Recovery’ procedure – needs BU
and TAC involvement)
ACI 4.1(1)
APIC Cluster Deployment Considerations Release
What about a 4 Nodes APIC Cluster?
Pod 1
X Pod 2
Up to 50 msec
X X
APIC APIC APIC APIC APIC
§ Intermediate scalability values compared to a 3 or 5 nodes cluster scenario (up to

170-200 leaf nodes supported)
§ Pod isolation scenario: same considerations as with 5 nodes (different behaviour
across shards)
§ Pod hard failure scenario
• No chance of total loss of information for any shard
• Can bring up a standby node in the second site to regain full majority for all the shards
APIC Cluster Deployment Considerations
Deployment Recommendations
§ Main recommendation: deploy a 3 nodes APIC cluster when less than 80 leaf nodes are
deployed across Pods
§ From ACI release 4.1(1) can deploy 4 nodes to support up to 200 leaf nodes across Pods
§ When 5 (or 7) nodes are really needed for scalability reasons, follow the rule of thumb of never
placing more than two APIC nodes in the same Pod (when possible):
Pod1 Pod2 Pod3 Pod4 Pod5 Pod6
2 Pods* APIC APIC APIC APIC APIC
3 Pods APIC APIC APIC APIC APIC
6+ Pods APIC APIC APIC APIC APIC
*’ID Recovery’ procedure possible for recovering of lost information

Multi-Pod and Virtual Machine Manager
(VMM) Integration
ACI Multi-Pod and VMM Integration
IPN
Pod 1 Pod 2
VMM Domain
DC1
Stretched HV Cluster
HV HV HV vSwitch1 HV HV HV
• Cluster of Hypervisors stretched across Pods

Ø Single VMM domain created across Pods
• Logical switch extended across the hypervisors part of the same stretched cluster
• Support for all intra-cluster functions (vSphere HA/FT, DRS, etc.)
ACI Multi-Pod
Connectivity to the External L3 Domain
Connecting Multi-Pod to Layer 3 Domain
Sharing L3Out Across Pods
§ A Pod does not need to have a dedicated WAN

connection (i.e. can offer transit services to other
MP-BGP - EVPN
Pods)
§ Multiple WAN connections can be deployed
across Pods
Pod 1 Pod 2
§ Outbound traffic: by default VTEPs always select
WAN connection in the local Pod based on
preferred metric WAN WAN
§ Leaf nodes in Pods without local L3Outs will Pod 3

load-balance traffic between L3Outs in remote Same IS-IS metric to
Pods reach BL nodes in Pod
1 and 2
ACI 4.0(1)
Connecting Multi-Pod to Layer 3 Domain Release
Use of Host-Route Advertisement on BL L3Outs
10.10.10.20/32 host
§ Host routes advertisement is a best route advertisement
option to ensure all the deployed FWs MP-BGP - EVPN
are actively utilized

• Support for host route advertisement on
BL nodes available from ACI release 10.10.10.10/24 10.10.10.20/24
Pod 1
4.0(1) 10.10.10.10/32 host
Pod 2
route advertisement
• Enabled at the BD level
WAN WAN
• Requires an L3Out connection in each
Pod Pod 3
• Allows to keep symmetric inbound and

outbound traffic paths
ACI Multi-Pod
Network Services Integration Models
Multi-Pod and Network Services Typical options for an
Active/Active DC use case
Integration Models
ISN
§ Active and Standby pair deployed across Pods

§ No issues with asymmetric flows
Active Standby
ISN • Active/Active FW cluster nodes stretched across Sites

(single logical FW)
• Requires the ability of discovering the same MAC/IP info
in separate sites at the same time
Active/Active Cluster
• Supported from ACI release 3.2(4d) with the use of
Service-Graph with PBR
ISN
§ Independent Active/Standby pairs deployed in separate
Pods
§ Use of Symmetric PBR to avoid the creation of
asymmetric paths crossing different active FW nodes
Active/Standby Active/Standby #CiscoLiveLA TECACI-2009 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 139
ACI Multi-Pod
Where to Go for More Information
ü ACI Multi-Pod White Paper

http://www.cisco.com/c/en/us/solutions/collateral/data-center-
virtualization/application-centric-infrastructure/white-paper-c11-
737855.html?cachemode=refresh
ü ACI Multi-Pod Configuration Paper

https://www.cisco.com/c/en/us/solutions/collateral/data-center-
virtualization/application-centric-infrastructure/white-paper-c11-739714.html
ü ACI Multi-Pod and Service Node Integration White Paper

ü BRKACI-2003
Agenda
Disaster Recovery Use Case “Disaster Avoidance (DA)” “Disaster Recovery (DR)”
§ Drive cost efficiencies through re-use of infrastructure and processes DC-1 DC-2 DC-3
§ Integrate Disaster Recovery into day to day operations

Active-Active

§ Make capacity growth sustainable through repurposed infrastructure

and shared resources leveraging virtualization
§ Provide a DR capability that is acceptable for any framework to all
subsidiaries (multi-tenancy)
§ Supports failover of passive services
o VMware Site Recovery Manager (SRM)
o Microsoft Cluster Services (MSCS)
o IBM HA Clustering Multi-proc (HACMP) / (PowerHA)
o Etc..
§ Global Server Load Balancing, Route Health Injection or LISP (Path
Redirection)
ACI Multi-Site ACI Multi-Site:
The Ideal Architecture for DR Deployments BRKACI-2125
VXLAN LABACI-2000
Inter-Site
Network
MP-BGP - EVPN
Multi-Site
Orchestrator
Site 1 REST
Site 2
GUI
API Availability Zone ‘B’
Availability Zone ‘A’
Region 1
• Separate ACI Fabrics with independent APIC clusters • MP-BGP EVPN control plane between sites
• No latency limitation between Fabrics • Data Plane VXLAN encapsulation across sites
• ACI Multi-Site Orchestrator pushes cross-fabric configuration to multiple • End-to-end policy definition and enforcement
APIC clusters providing scoping of all configuration changes
ACI Multi-Site
Hardware Requirements
§ Support all ACI leaf switches (1st

Generation, -EX and -FX)
Can have only a subset
§ Only –EX spine (or newer) to connect to IP Network
of spines connecting to
the IP network
the inter-site network
§ New 9364C non modular spine 1st Gen 1st Gen -EX -EX
(64x40G/100G ports) supported for Multi-
Site from ACI 3.1 release (shipping)
§ 1st generation spines (including 9336PQ)
not supported
Can still leverage those for intra-site leaf to leaf
communication
ACI Multi-Site
Network and Identity Extended between Fabrics
Network information carried across Identity information carried across
Fabrics (Availability Zones) Fabrics (Availability Zones)
VTEP IP VNID Class-ID Tenant Packet No Multicast Requirement in

Backbone, Head-End
Replication (HER) for any
IP Network Layer 2 BUM traffic)
MP-BGP - EVPN
ACI Multi-Site
Inter-Site Policies and Spines’ Translation Tables
§ Inter-Site policies defined on the ACI IP

Network
Multi-Site Orchestrator are pushed to
the respective APIC domains
• End-to-end policy consistency
• Creation of ‘Shadow’ EPGs to locally
represent the policies
§ Inter-site communication requires the
installation of translation table entries on EP1 EP2
the spines (namespace normalization) Site 1 Site 2
§ Up to ACI release 4.0(1) translation EP1

EPG C EP2 EPG
EP1
EPG C EP2 EPG
entries are populated only in two cases:

1. Stretched EPGs/BDs
‘Shadow’
2. Creation of a contract between not EPGs
stretched EPGs

ACI Multi-Site Networking Options Should be the behavior for the
Per Bridge Domain Behavior majority of BDs with Multi-Site
Layer 3 only across sites IP Mobility without BUM flooding Layer 2 adjacency across Sites
1 2 3
ISN ISN ISN
Site Site Site Site 2
Site Site Site
1 2 1 2 1 2
§ Bridge Domains and subnets not § Same IP subnet defined in separate § Interconnecting separate sites for
extended across Sites Sites fault containment and scalability
Support for IP Mobility (‘cold’ and reasons
§ Layer 3 Intra-VRF or Inter-VRF §
communication (shared services ‘live’* VM migration) and intra- § Layer 2 domains stretched across
across VRFs/Tenants) subnet communication across sites Sites, support for application
§ No Layer 2 BUM flooding across clustering
sites § Layer 2 BUM flooding across
sites
MSO GUI MSO GUI MSO GUI

(BD) (BD) (BD)
*’Live’ migration officially supported from ACI release 3.2

ACI Multi-Site
Multi-Site Orchestrator, Schemas and Templates
ACI Multi-Site
Multi-Site Orchestrator (MSO)
• Three MSO nodes are clustered and run concurrently (active/active)
§ Typical database redundancy considerations
(minority/majority rules)
REST § Up to 150 msec RTT latency supported between MSO nodes
GUI
API
§ vSphere VM only form factor initially, physical appliance
planned for a future ACI release
ACI Multi-Site Orchestrator
150 msec RTT • OOB Mgmt connectivity to the APIC clusters deployed in
(max)
VM VM VM separate sites
§ Up to 1 sec RTT latency between MSO and APIC nodes
Hypervisor
• Main functions offered by MSO:
1 sec RTT § Monitoring the health-state of the different ACI Sites
(max)
§ Provisioning of day-0 infrastructure configuration to establish
inter-site EVPN control plane and VXLAN data plane
…..
Site 1 Site 2 Site n § Defining and provisioning tenant policies across sites
§ Day-2 operation functionalities
ACI Multi-Site
MSO Schema and Templates
Schema
§ Template = ACI policy definition
(ANP, EPGs, BDs, VRFs, etc.)
§ Schema = container of Templates
sharing a common use-case
• As an example, a schema can be dedicated
to a Tenant
§ The template is currently the atomic
unit of change for policies
• Such policies are concurrently pushed to
one or more sites Site 1 Site 2
§ Scope of change: policies in different EFFECTIVE
POLICY
EFFECTIVE
POLICY
templates can be pushed to
separate sites at different times

ACI Multi-Site
Schema and Templates Definition for the DR Use Case
Future
Schema Schema Schema
Template 1 Template 1 Template 2 Template 1

EP1 EP2 EP1 EP2 EP1 EP2 EP1 EP2
C C C C
EPG EPG EPG EPG EPG EPG EPG EPG
t1 t1 t1 t2 t1 t2
Prod Site DR Site Prod Site DR Site Prod Site DR Site
§ Single Template associated to Prod § Separate Template associated to Prod § Single Template associated to Prod
and DR Sites and DR Sites and DR Sites
§ Any change applied to the template § Changes made to a template can be § Capability of independently apply
is pushed to both sites applied only to the mapped site changes to each site
simultaneously
§ Requires sync between the two § Brings together the advantages of
§ Easiest way to keep consistent templates (manual or performed by an the previous two options
policies deployed across sites higher level Orchestration
#CiscoLiveLA
tool) 151
TECACI-2009
Multi-Site and Virtual Machine Manager
(VMM) Integration
ACI Multi-Site and VMM Integration
Option 1 – Separate VMM per Site
ISN
Site 1 VMM Domain VMM Domain Site 2

DC1 DC2
VMM 1 VMM 2
HV vSwitch1
HV HV Managed by HV vSwitch2
HV HV
VMM 1
HV Cluster 1 Managed HV Cluster 2
by VMM 2
• Typical deployment model for an ACI Multi-Site

• Creation of separate VMM domains in each site, which are then exposed to
the Multi-Site Orchestrator
ACI Multi-Site and VMM Integration
Workload Migration across Sites
ISN
Site 1 VMM Domain VMM Domain Site 2

DC1 DC2
vCenter vCenter
Server 1 Server 2
SRM SRM
HV HVVDS1 HV
EPG1 HV HVVDS2 HV
EPG1
HV Cluster 1 HV Cluster 2
Live vMotion/Cold Migration

• Live virtual machines migration across sites is supported only with vCenter deployments
(both for single or multiple vCenter options)
§ Requires vSphere 6.0 and newer, no support for DRS, vSphere HA/FT
• Use of Site Recovery Manager (SRM) or similar higher level orchestrator for workload
recovery across sites
ACI Multi-Site
Connectivity to the External L3 Domain
ACI 4.2(1)
ACI Multi-Site and L3Outs Release
Support of Inter-Site L3Out
• Starting with ACI Release 4.2(1) it is possible for

Inter-Site Network endpoints in a site to send traffic to resources
(WAN, Mainframes, etc.) accessible via L3Out
of a remote site
• Traffic will be directly encapsulated to the TEP
of the remote BL nodes
• The BL nodes will get assigned an address part of an
L3Out L3Out
Site 1 Site 2 additional (configurable) prefix that must be routable
across the ISN
WAN
• Same solution will also support transit routing
across sites (L3Out to L3Out)
Multi-Site and L3Out
Endpoints always Use Local L3Outs for Outbound Traffic
Inter-Site Network
Site 1 Site 2
Web-EPG C1 Ext-EPG
L3Out L3Out
Site 1 Site 2
10.10.10.10 IP Subnet 10.10.10.11
IP Subnet Active/Standby
10.10.10.0/24 Active/Standby
10.10.10.0/24
Traffic dropped
because of lack of
state in the FW
Multi-Site and L3Out ACI 4.0(1)
Release
Use of Host-Routes Advertisement
Inter-Site Network
Site 1 Site 2
Web-EPG C1 Ext-EPG
L3Out L3Out
Site 1 Site 2
10.10.10.10 Host routes 10.10.10.11
10.10.10.10/32
Active/Standby Active/Standby Host routes
10.10.10.11/32
*Alternative could be
running an overlay solution
Host-routes
(LISP, GRE, etc.) injected into the
WAN* Enabled at
the BD level
• Ingress optimisation requires host-routes advertisement on the L3Out
§ Native support on ACI Border Leaf nodes available from ACI release 4.0
§ Supported also on GOLF L3Outs #CiscoLiveLA TECACI-2009 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 158
Multi-Site and L3Out ACI 4.2(1)
Release
Active/Standby FW Deployed across Sites
Inter-Site Network
Site 1 Site 2
Web-EPG C1 Ext-EPG
L3Out L3Out
Site 1 Site 2
10.10.10.10 10.10.10.11
Active Standby
• Inbound and outbound flows are forced through the site with the active perimeter FW node
• Mandates inter-site L3Out support (ACI release 4.2)
ACI Multi-Site
Network Services Integration
Multi-Site and Network Services Deployment options fully
supported with ACI Multi-Pod
Integration Models
ISN
• Active and Standby pair deployed across Pods

• Currently supported only if the FW is in L2 mode or in L3
mode but acting as default gateway for the endpoints
• From ACI 4.2(1) will be also supported as perimeter FW
Active Standby
ISN
• Active/Active FW cluster nodes stretched across Sites
(single logical FW)
• Requires the ability of discovering the same MAC/IP info in
separate sites at the same time
Active/Active Cluster
• Not currently supported (scoped for a future ACI
release)
ISN • Recommended deployment model for ACI Multi-

Site
• Option 1: supported from 3.0 for N-S if the FW is
connected in L3 mode to the fabric à mandates the
deployment of traffic ingress optimization
Active/Standby Active/Standby • Option
#CiscoLiveLA 2: supported
TECACI-2009 from
© 2019 3.2itsrelease
Cisco and/or with
affiliates. All rights theCisco
reserved. use of
Public 161
Service Graph with Policy Based Redirection (PBR)

ACI Multi-Site
Integration with ACI Multi-Pod
ACI Multi-Pod and Multi-Site ACI 3.2(1)
Release
Main Use Cases
§ Adding a Multi-Pod Fabric as a ‘Site’ on the Multi-Site Orchestrator (MSO)
§ Converting a single Pod Fabric (already added to MSO) to a Multi-Pod fabric
ACI Multi-Pod and Multi-Site
Connectivity between Pods and Sites
IP WAN
IPN
Site 2
1st Gen 1st Gen
APIC Cluster
Pod ‘A’ Pod ‘B’
Site 1 Site 2
§ Only 2nd generation spines must be connected to the external network

• Need to add 2nd gen spines in each Pod (at least two per Pod) and migrate connections to the IPN from 1st gen
spines to 2nd gen spines
§ Single ‘infra’ L3Out and set of uplinks to carry both Multi-Pod and Multi-Site East-West traffic
Connectivity between Pods and Sites
Not Supported Topology
Separate uplinks
between spines IP WAN
and external
networks
IPN
Site 2
1st Gen 1st Gen
APIC Cluster
Pod ‘A’ Pod ‘B’
Site 1 Site 2
§ Only 2nd generation spines must be connected to the external network

• Need to add 2nd gen spines in each Pod (at least two per Pod) and migrate connections to the IPN from 1st gen
spines to 2nd gen spines
§ Single ‘infra’ L3Out and set of uplinks to carry both Multi-Pod and Multi-Site East-West traffic
ACI Multi-Site
ü ACI Multi-Site White Paper

ü Deploying ACI Multi-Site from Scratch

https://www.youtube.com/watch?v=HJJ8lznodN0
ü BRKACI-2125
Agenda
ACI Remote Physical Leaf ACI Remote Leaf:
Business Value and Use Cases BRKACI-2387
Extending the ACI policy model outside the main datacenter to remote
sites distributed over IP Backbone (Telco DCs, CoLo locations, etc.)
Extending ACI fabric policy and L2/L3 connectivity to a small DC site

without requiring the deployment of a full-blown ACI Fabric or for
migration/coexistence with legacy DC sites
Centralized Policy Management and Control Plane for remote locations
Small form factor solution at locations with space constraints
ACI Remote Physical Leaf
Migration/Coexistence Use Case
Legacy DC Site
IP WAN
Layer 2
Layer 2 RL Pair
• Connecting a greenfield ACI fabric to a legacy DC location

§ Single point of management
§ Coexistence
§ Application migration
‘Disaggregated DC Model” for 5G Deployment
• Increased SP DC Footprint with edge transformation, vRAN
• Application/Services are distributed(CUPS, CU-DU split etc)
Central DC Regional DC Edge DC Far Edge
IP Core/Edge Metro Backhaul
Internet
DC Infrastructure DC Infrastructure DC Infrastructure

VM VM
170
ACI 3.1(1)
ACI Remote Physical Leaf Release
Conceptual Architecture
A Remote Leaf ‘Site’ gets
associated with the Spines of
VXLAN
one specific Pod in Main DC
APIC and Spine Nodes (Proxy
function) remain at primary Pod(s)
IP WAN L2 / L3
IP WAN Requirements (4.1(1))

§ 300 msec maximum RTT
§ 100 Mbps minimum BW
§ 1600B minimum MTU
§ No PIM-Bidir required vSwitch Bare
Metal
Legacy
Infrastructure
Hypervisor
ACI Main DC
Remote Leaf Site: a pair of Nexus 9300 nodes

connected to a L3 Network via uplink ports and
fully managed by a centralized APIC cluster
Hardware/Software Support
ACI Main DC Remote Location
Supported Spines Supported Leaf

Fixed • All 2nd Gen ACI Leafs
• 9364C/9332C
Modular
• 9732C-EX
• 9736C-FX
All hardware from -EX onwards is supported
Remote and Local Endpoints Communication
COOP DB
EP1 RL TEP
EP2 RL TEP IP WAN IPN Anycast

Anycast VTEP for
VTEP for RLs
RLs (RL TEP)
EP3 L4-TEP (RL TEP)
RL Local Table
Ucast-TEP
EP1 vPC Po1
Po1 Po2 EP2 vPC Po2
EP3 Ucast-TEP
EP1 EP2
Remote Leaf Site

EP3
ACI Main DC Learned via data-plane
(not COOP)
Hair-Pin Communication between RL Pairs RL Local Table
Remote Leaf Pair 2
EP1 Ucast-TEP
Current behavior as ACI release 4.1(1) EP3
EP2 Ucast-TEP
COOP DB Po1 EP3 vPC Po1
EP1 RL-TEP1
RL-TEP2
EP2 RL-TEP1 IP WAN IPN
EP3 RL-TEP2
RL Local Table
Ucast-TEP
RL-TEP1
EP1 vPC Po1
Po1 Po2 EP2 vPC Po2
EP3 Ucast-TEP
EP1 EP2
Remote Leaf Pair 1

ACI Main DC
ACI 4.1(2)
Direct Communication between RL Pairs RL Local Table
Remote Leaf Pair 2
EP3 EP1 RL-TEP1
EP2 RL-TEP1
COOP DB Po1 EP3 vPC Po1
EP1 RL-TEP1
RL-TEP2
EP2 RL-TEP1 IP WAN IPN
EP3 RL-TEP2
RL Local Table
Ucast-TEP
RL-TEP1
EP1 vPC Po1
Po1 Po2 EP2 vPC Po2
EP3 RL-TEP2
EP1 EP2
Remote Leaf Pair 1

ACI Main DC
Outbound Traffic Flows
IP WAN IPN
RL1 RL2
TEP TEP
RL Local Table
Endpoints use
the local L3Out
20.20.20.0/24 Local L3Out
for outbound
traffic BL1 BL2
TEP TEP RL L3Out
Endpoint uses the

Pod L3Out local L3Out for
outbound traffic
L1 Local Table ACI Main DC Remote Leaf Site

BL1, BL2
20.20.20.0/24
TEPs 20.20.20.0/24
Outbound Traffic Flows (L3Out Failure)
IP WAN IPN
L1 Local Table RL Local Table

20.20.20.0/24 BL1, BL2 20.20.20.0/24 BL1, BL2
TEPs TEPs
BL1 BL2
TEP TEP RL L3Out
Start using
external route
learned from
Pod L3Out
Main DC
ACI Main DC Remote Leaf Site

20.20.20.0/24
Inbound Traffic Flows
Sub-optimal
inbound
traffic flow
IP WAN IPN
RL1 RL2
TEP TEP
BL1 BL2
TEP TEP RL L3Out
10.10.10.11/24
10.10.10.0/24
Pod L3Out
10.10.10.0/24
10.10.10.10/24 ACI Main DC Remote Leaf Site
ACI 4.0(1)
Inbound Traffic Flows with Host Routes Advertisement
IP WAN IPN
RL1 RL2
TEP TEP
BL1 BL2
TEP TEP RL L3Out
10.10.10.11/24
10.10.10.11/32
Pod L3Out
10.10.10.10/32 Optimized
10.10.10.10/24 ACI Main DC inbound Remote Leaf Site
traffic flows
ACI Remote Physical Leaf and Multi-Pod
RL Sites Can Be Associated to Separate Pods
RL Site 2
RL Site 1 associated
associated to Pod2
to Pod1
RL Site1 RL Site 2
Inter-Pod Network IP WAN
IP WAN
(Multicast Enabled)
ACI Main DC Pod1 ACI Main DC Pod2
ACI Remote Physical Leaf and Multi-Pod
RL Sites Can Be Associated to Separate Pods (Data Plane)
RL Site 1 RL Site 2
IP WAN
(Multicast Enabled)
ACI 4.1(2)
ACI Remote Physical Leaf and Multi-Pod Release
RL Sites Can Be Associated to Separate Pods (Data Plane)
Direct communication between RL nodes associated

to different Pods
RL Site 1 RL Site 2
IP WAN
(Multicast Enabled)
ACI 4.1(2)
ACI Remote Physical Leaf and Multi-Site Release
RL Site 2
associated to ACI Site 2
to ACI Site 1
RL Site 1 RL Site 2
IP WAN
IP WAN Inter-Site Network
ACI Site 1 ACI Site 2
ACI 4.1(2)
ACI Remote Physical Leaf and Multi-Site Release
RL Site 2
associated to ACI Site 2
to ACI Site 1
RL Site 1 RL Site 2
IP WAN
IP WAN Inter-Site Network
ACI Site 1 ACI Site 2
ACI Physical Remote Leaf
ü ACI Remote Physical Leaf White Paper

https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-
infrastructure/white-paper-c11-740861.html
ü BRKACI-2387
Agenda
Baremetal Cloud
ACI Virtual Pod (vPod) vSpine

Virtual Pod
vSpine
Main Use Cases Baremetal cloud
vTOR vTOR
ACI Virtual Edge
Brownfield
Virtual Pod
vSpine vSpine
IP Network
vTOR vTOR
ACI Virtual Edge
Colocation / Remote DC
Virtual Pod
On-Premises ACI Data Center vSpine vSpine
For More Information on vTOR vTOR

ACI Virtual Pod:
ACI Virtual Edge
BRKACI-2882
ACI Virtual Pod
Extend ACI to Brownfield or Baremetal Cloud Locations
Legacy DC Site / Baremetal Cloud
IP WAN
Layer 2
Layer 2
AVE AVE AVE
Hypervisor Hypervisor Hypervisor
ACI “Virtual Leaf” Nodes
Switching/routing and
policy enforcement
• vPod allows to extend ACI connectivity and policies to compute resources deployed
in legacy DC networks
• No need to deploy any ACI HW in the remote network
What vPod is NOT Today
Virtual ACI Fabric

Physical fabric and APIC
cluster are still required in
the main Datacenter An ACI site
vPod is a new Pod in ACI
SDN for Public Cloud

vPod operates in the same
VXLAN namespace as the
physical fabric
ACI 4.0(2)
ACI Virtual Pod Release
Architectural Components
Management Cluster (vSpine + vLeaf)

• vSpine nodes: centralized endpoint and LPM
Virtual Pod database (COOP and BGP)
• vLeaf nodes: distribute APIC policies to ACI Virtual

Edges (DME/PE on vLeaf <-> Opflex on AVE)
vSpine vSpine
• vSpine and vLeaf nodes are not used for data-plane
Control Plane
forwarding
ACI Virtual Edge (vPod Mode)

vLeaf vLeaf • Implements ACI data plane function (switching and
routing) and policy enforcement
ACI Virtual Edge Data Plane • iVXLAN for communication within vPod and across
Pods
ACI Virtual Pod
Software and Hardware Requirements
On-Premises Datacenter vPod Datacenter
Supported Spines
• VMware vCenter 6.0 or later
Fixed Spine:
• 2 hosts for Management Cluster
• N9364C
• N9332C • 2 hosts for Payload Cluster
• ESXi 6.0 or later

Modular Spine LC: (C9504/C9508/C95016)
• N9732C-EX with FM N9K-C950x-FM-E(2) • ACI vCenter plugin or vPod
• N9736C-FX with FM N9K-C950x-FM-E(2) python/PowerShell deployment scripts
• APIC 4.0(2) onwards
ACI Virtual Pod
Control and Data Planes
Mandates the deployment of VXLAN
2nd gen spines (EX or newer)
Data Center A
Virtual Pod
MP-BGP - EVPN
IP Network
vSpine vSpine
vLeaf vLeaf
ACI Virtual Edge
COOP, MP-BGP
IS-IS, COOP, MP-BGP
On-Premises ACI Data Center

• Policies centrally defined on the APIC cluster deployed on-prem
• MP-BGP EVPN sessions established to exchange endpoint reachability information
between Pods
• Ingress replication support on physical spines and AVEs to forward BUM traffic
ACI Virtual Pod
Inter-Pod Network (IPN) Requirements
Data Center A
Virtual Pod
MP-BGP - EVPN
IP Network
vSpi ne vSp ine
150 msec RTT

v Leaf vLeaf
ACI Virtual Edge
COOP, MP-BGP
IS-IS, COOP, MP-BGP
§ Not managed by APIC, must be pre-configured (day-0 configuration)

§ IPN topology can be arbitrary, not mandatory to connect to all physical spine nodes
§ Main requirements:
ü OSPF to peer with the physical/virtual spines and learn VTEP reachability information
ü Increased MTU support (at least 50 extra Bytes) to handle VXLAN encapsulated traffic
ü DHCP-Relay
ü Latency up to 150 msec RTT
ACI Virtual Pod
ü Virtual Pod White Paper https://www.cisco.com/c/en/us/solutions/collateral/data-

center-virtualization/application-centric-infrastructure/white-paper-c11-742393.html
ü BRKACI-2882
Agenda
Cloud ACI
Multi-Site Orchestrator
2HCY19
Cloud Region(s) On-Premises Cloud Region(s)
#CiscoLiveLA © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 196
ACI Extensions to Cloud Multi-Site
On-Premises DC Public Cloud
SG SG SG
SG Rule SG Rule
Web APP DB
EPG EPG EPG
Contract Contract
Web APP DB
AWS Region
ASG ASG ASG

NSG NSG
Web APP DB
Azure Region
Consistent Policy Enforcement Automated Inter-connect Simplified Operations

on-Premises & Public Cloud provisioning with end-to-end visibility
Use Cases
TECACI-2009 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 198
Supported
ACI 4.1
Application Stretch
On-Premises Public Cloud
• Stretch tenant/VRF across on-

APIC Cloud APIC premises and cloud sites
Tenant
• During peak times easily deploy
VRF application tiers and resources in the
BD1/Subnet CIDR 2
1Web-EPG1 Web-EPG2
cloud site
• Consistent segmentation policy and

HTTPs HTTPs enforcement within and across on-
premises and cloud sites
BD3/Subnet3 CIDR 4
App-EPG1 App-EPG2
• Application stack failover between
sites (active/disaster recovery)
Supported
ACI 4.1
Stretched EPG with Consistent Segmentation
APIC Cloud APIC

• Web Tier and App Tier are stretched
Tenant and securely segmented across on-
VRF
premise and public cloud sites
BD/Subnet1 CIDR 2
EPG - Web • Consistent segmentation policy and
enforcement for endpoints of
HTTPs, redis
Web/App Tier are independent of
location
BD3/Subnet3 CIDR 4
EPG - App
What? I can stretch EPGs to AWS? You mean I can stretch
VLANs???

No really, can I stretch VLANs to the cloud??
• Absolutely not!
• Stretching an EPG does not mean stretching a VLAN or broadcast domain.
• Stretched EPG Foo uses subnet X on-prem and subnet Y in the cloud
• You tell MSO what criteria(s) should be used to join EPG Foo
• Could be tags, could be an IP prefix, could be a region or an AZ
• cAPIC informs MSO when an EC2 instance matches the selector
• EC2 instance appears as a /32 in external EPG on-prem, shadow EPG takes care of
contracts to make it look like a stretched EPG
• cAPIC programs a security group in AWS to match on-prem contracts
• Anyway, public cloud vendors do not allow broadcast or multicast.

• There is no good use case for this.

Supported
ACI 4.1
Shared Services for Hybrid-Cloud

• Provides a capability to
APIC Cloud APIC deploy shared service
across hybrid cloud
Tenant 1 Route Tenant 2 Tenant 3
Leaking
VRF2 VRF3 • Shared Service
VRF1
CIDR 2 CIDR 4
deployed in 1 Site can
DNS Web-EPG Web-EPG be consumed by
endpoints across other
BD/Subnet1
HTTPs HTTPs, redis
sites
DNS-EPG
CIDR 3 CIDR 5
• Contract will leak
App-EPG App-EPG subnet between VRFs
for reachability
Supported
ACI 4.1
Cloud and On-Prem L3outs
Multi-Site Orchestrator (MSO)
On-Premise Public Cloud
Site
• Cloud local L3out via

IGW
L3out
• On-Prem local L3out

Site
• On-Prem site
endpoints cannot use
Cloud L3out
EPG-1 EPG-1 EPG-2 EPG-3
• Shared On-Prem L3out
IGW IGW
L3out
for Cloud VPCs *
L3out
SG-1 SG-1 SG-2 SG-3
Supported
ACI 4.2
Cloud First
• Cloud APIC only without on-premises ACI
• Optional MSO
• Abstract AWS and/or Azure networking
constructs from user that is familiar with
ACI, delivering ACI-consistent policy and
MSO
operational model
• Deploy EPG and contracts on top of AWS
and/or Azure public cloud
Cloud Infra – AWS
Multisite Orchestrator
ACI DC
Region - 1
Infra VPC
IPSec Tunnel
User VPC 1 User VPC 2
Cloud Infra – Azure
ACI DC
Region - 1
Infra VNET
IPSec Tunnel
User VNET 1 User VNET 2
#CiscoLiveLA © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public
Supported
ACI 4.1
Virtual Private Network (VPN)
On-Premise Public Cloud
Site
Site
User VPC-1
IPSec VPN Tunnel (Underlay)
CSR1000V
VGW
BGP-EVPN Session (Control Plane)
VXLAN Tunnel (Data Plane)
Customer
Premise AWS
AWS Instances
Router
Internet Internet
Gateway
Infra VPC VGW
AWS Instances
AWS Region User VPC-2
• VXLAN data-plane connects ACI fabric and Cloud site

• BGP-EVPN routing reachability between ACI fabric and Cloud Site
• IPSec VPN connection between customer Premise Router before ACI fabric and CSR1kv
AWS Direct Connect (DX) / Azure ExpressRoute
On-Premise
Site Public Cloud Site
User VPC-1
VXLAN VGW
BGP-EVPN
Direct Connect (DX) / BGP Underlay CSR1000V AWS Instances
Border Amazon
ACI Leaf DGW/
VGW
Infra VPC
AWS Region VGW
• Direct Connect and BGP underlay between Infra-VPC and

ACI Border Leaf AWS Instances
• BGP-EVPN and VXLAN over Direct Connect ACI fabric to User VPC-2
CSR 1000v
ACI Anywhere extenstion to public cloud:
components required
• Three main components:
• On-prem ACI
• 2nd generation spines required (like regular multi-site)
• MSO 2.1 (hosted on-prem)
• Same IPN connectivity as traditional multi-site
• You need an IPN layer (OSPF w/Spines)
• And an on-prem CSR1Kv or ASR to terminate IPsec
• MP-BGP eVPN between Spines and CSRs in the cloud
• AWS site (cloud APIC)
• Infra VPC (where cAPIC and CSRs reside)
• Tenant VPC (only AWS components)

Wait, that looks quite complex
• You don’t configure half of what you just saw
• You spin up a cAPIC using a Cloud Formation Template
• Pair on-prem with your IPN just like regular multi-site
• MSO and cAPIC take care of most configuration aspects
• You provide high-level config parameters
• You get a ready-to-use IPsec configuration to copy/paste in your CSRs
• All AWS configuration aspects (VGW BGP and IPsec, routing, security groups) is
fully automated and abstracted

Cloud APIC: what is it?
Cloud APIC Architecture
• Virtual Form Factor of APIC
• Automates / Manages Cloud Routers
Web Server (NGINX)
• Translates ACI Policy to cloud native constructs
Policy Distributor (PD)
• Deploys cloud resources and infrastructure
Policy Manager (PM) components
Cloud Policy Cloud Policy • Intuitive GUI and Similar ACI UI look and feel
Element Element
….
Connector Connector
• REST API North Bound Interface
API (AWS, Azure...) • cAPIC manages 1 or more regions

NetConf (CSR1000v)
For your
info &
Policy Mapping - AWS

reference
User Account Tenant

Virtual Private Cloud VRF
VPC subnet BD Subnet
Tag / Label EP to EPG Mapping

Security Group EPG
Network Access List Taboo
Security Group Rule Contracts, Filters
Outbound rule
Consumed contracts
Source/Destination: Subnet or IP or Any or ‘Internet’
Protocol
Port
Inbound rule Provided contracts
EC2 Instance
Network Adapter End Point (fvCEp)
For your
info &
Policy Mapping - Azure

reference
Resource Group Tenant

Virtual Network VRF
Subnet BD Subnet
Application Security Group EPG

(ASG)
Network Security Group

(NSG) Filters
Outbound rule Consumed contracts

Source/Destination: ASG or Subnet or IP or Any or ‘Internet’
Protocol
Port
Inbound rule Provided contracts

Virtual Machine
Network Adapter
End Point Learning in Cloud
Region 1 • User deploys a new

instance in AWS
Cloud APIC
Infra VPC • AWS config services
AWS config notifies the event to cAPIC
services
• cAPIC learns the endpoint
and registers it
• Based on the policies
(EPG’s and Contracts) the
correct security group
(SG) is attached to the
instance
SG-1
Security Group (SG)
Availability Zone (AZ)
Cloud EPG
Mapping Endpoints by Tags WEB EPG DB EPG
Site B • Web-EPG associated
to tag: “EPG: WEB”
• Web-EPG has
endpoints across
Us-East-1 & Us-West-
1 regions and multiple
subnets
Subnet-S1 – 10.1.1.0/24 Subnet-S3 – 10.1.3.0/24
• DB-EPG associated to
tag: “EPG:DB”
• DB-EPG has endpoints

across Us-East-1 &
Subnet-S2 – 10.1.2.0/24 Subnet-S4 – 10.1.4.0/24
Us-West-1 regions and
multiple subnets
US-East-1 US-West-1

Deploying
Cloud APIC
TECACI-2009 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 218
Cloud APIC in AWS Marketplace http://cs.co/capic-aws
Cloud APIC in Azure Marketplace http://cs.co/capic-azure
Agenda

ACI Anywhere

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ACI Anywhere

Uploaded by

Copyright:

Available Formats

ACI Fabric Design

Choose the type of ACI fabric you want to implement

• ACI 2-tier fabric:

• ACI 3-tier fabric: Physical APIC

More about ACI Mini at this link:

• Define which scale profile

• The scalability is the same as a 2-tier Spines

• First generation leafs can be part of the

• Please refer to the documentation for the

• All ports of Tier-1 Leafs can be

• Tier-2 Leaf fabric ports connect to Tier-

• APIC, L3out, EPGs can be connected to

If APIC is connected to a tier-2 leaf:

If no APIC is connected to a tier-2 leaf:

FC Initiator FCoE Initiator

RR RR MP-BGP EVPN with Nexus 7k or ASR9k

Border Leaf (s)

BD subnets for VRF1 BD subnets for VRF2

More about the L3out later

Network Fault/Change/Reachability Domain Network Fault/Change/Reachability Domain

Reachability is Decoupled from Topology as well

#CiscoLiveLA TECACI-2009 113

Metro Area Pervasive Data Protection

MP-BGP - EVPN MP-BGP - EVPN

ACI Physical Remote Leaf ACI Virtual Remote Leaf (vPod)

That is the question…

• Regions – Separate large geographical areas, each Amazon Web Services

Fabric ‘A’ (AZ 1)

Fabric ‘B’ (AZ 2)

Multi-Pod Fabric ‘A’ (AZ 1)

Pod ‘1.A’ Pod ‘2.A’

Multi-Pod Fabric ‘B’ (AZ 2)

Pod ‘1.B’Application Pod ‘2.B’

§ High-availability application and data solution architecture which

coupled DC for short distances)

§ Management and interaction of applications in a paired data

Tenant ‘Prod’ Configuration/Change Domain

§ True Active/Active DC deployments

One copy is ‘active’ for every

Shard 1 Shard 1 Shard 1

Shard 2 Shard 3 Shard 2 Shard 3 Shard 2 Shard 3

• All Services in ACI run against their own portions of a Database

Read/Write Read Only

§ Intermediate scalability values compared to a 3 or 5 nodes cluster scenario (up to

Pod1 Pod2 Pod3 Pod4 Pod5 Pod6

2 Pods* APIC APIC APIC APIC APIC

3 Pods APIC APIC APIC APIC APIC

4 Pods APIC APIC APIC APIC APIC

5 Pods APIC APIC APIC APIC APIC

6+ Pods APIC APIC APIC APIC APIC

*’ID Recovery’ procedure possible for recovering of lost information

• Cluster of Hypervisors stretched across Pods

§ A Pod does not need to have a dedicated WAN

§ Leaf nodes in Pods without local L3Outs will Pod 3

option to ensure all the deployed FWs MP-BGP - EVPN

are actively utilized

• Allows to keep symmetric inbound and

§ Active and Standby pair deployed across Pods

ISN • Active/Active FW cluster nodes stretched across Sites

ü ACI Multi-Pod White Paper

ü ACI Multi-Pod Configuration Paper

ü ACI Multi-Pod and Service Node Integration White Paper

§ Integrate Disaster Recovery into day to day operations

Metro Area Pervasive Data Protection