Brkaci 2301

BRKACI-2301
Practical Applications of Cisco

ACI Micro Segmentation
Andy Sholomon
@asholomon, Principal Engineer – DC Switching
Session Objectives
• Provide an overview of what we mean when we talk about
implementing Micro Segmentation
• Describe the ACI features that help deploying Micro Segmentation
• Deep dive on what you can do with Micro EPGs and ACI contracts
• Provide ideas of how to use these features
• Show these features working on simple yet practical examples
• Show an example of Tetration and explain how it can work with ACI
BRKACI-2301 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Some comments about this deck
• We will cover some of the topics via examples and showing demos
• In the deck there are links to the code and videos of the demos
• Some slides are provided for your reference only, we may not talk through them.
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
cs.co/ciscolivebot#BRKACI-2301
Agenda
• Micro Segmentation Fundamentals
• Endpoint Identity Using EPGs and Micro EPGs (uEPG)
• ACI Contracts for Policy Definition
• Improvements in Hardware Utilization
• ACI and Hybrid Cloud Security
• Demo
What do we mean by
Micro Segmentation?
Why Micro Segmentation?
DC Perimeter
• Perimeter security is not enough: once

it is breached, lateral movement can allow
attackers to compromise more assets
• Micro Segmentation can improve the security posture inside the
Data Center
• Micro Segmentation can minimize segment size and provide lesser
exposure for lateral movement
Security Risk: VMs or Servers on a Single Subnet
172.16.10.11 172.16.10.12 172.16.10.13

VM1 VM2 VM3
Site1 Site2 Priv
Web Servers
172.16.20.11 172.16.20.12
VM4 VM5
App Servers
172.16.30.11 172.16.30.12
VM6 VM7
DB Servers
Security Risk: VMs or Servers on a Single Subnet
172.16.10.11 172.16.10.12 172.16.10.13

VM1 VM2 VM3
Web Servers Very simple

Micro Segmentation
use case
172.16.20.11 172.16.20.12
VM4 VM5
App Servers
172.16.30.11 172.16.30.12
VM6 VM7
DB Servers
A Micro Segmentation Use Case
172.16.10.11 172.16.10.12 172.16.10.13 172.16.10.14 172.16.10.15 172.16.10.16

VM1 VM2 VM3 VM4 VM5 VM6
172.16.10.0/24
Prod QA Dev
Another Micro Segmentation Use Case
172.16.10.11 172.16.10.12 172.16.10.13 172.16.10.14 172.16.10.15 172.16.10.16

VM1 VM2 VM3 VM4 VM5 VM6
172.16.10.0/24
Web App DB
Micro Segmenting in an Heterogeneous Data Center
Campus
and Many different types of workloads running in a Data Center
Branch Users
Micro Segmenting in an Heterogeneous Data Center
Virtualized w/ Virtualized w/
KVM Microsoft
Campus
and
Branch Users
Virtualized w/ VMware Bare Metal / Big Data Shared/Infra
Micro Segmenting requires granularly grouping endpoints,
and defining and enforcing policy between them
Sales
Contractor
Policy Enforcement
16
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public
Key Functions to Achieve Micro Segmentation
Endpoint Identity Policy Definition Verify and Refine
Classify endpoints into Determine what policy to Verify policy enforcement,

groups: configure between groups: lifecycle management:
- Network identity - Application Dependency - Policy visibility

(IP/MAC/VLAN) Mapping - Logging and log analysis
- Meta-data: VM attributes, - White-List vs. Black-List - Alerts, remediation
labels, tags, etc. - Policy Simulation - Constant updates
- DNS - Dynamic vs. pre-defined
- User Authentication (i.e.
from ISE)
Enforcement Points
Host-based - Centrally manage host-based firewalls
• Pros: distributed, network independent, very granular policies possible, process-
level visibility and correlation
• Cons: Guest-OS dependent
Network-based - Centrally manage access rules at the network edge (Virtual

Switch, Physical Switch or both)
• Pros: distributed, guest independent, group based policies for best scale,
endpoint-level visibility and correlation
• Cons: requires network hardware resources (memory, TCAM, etc) for policy
Cisco Tetration – cloud workload protection
 Real time
 Thousands of
workloads
 On premises
Micro
Segmentation
Data Leakage
Detection
and public
and cloud
ADM
 All types of
Integrity workloads
from
Vulnerability Management
Exploit
Assessment (process, file)
Detection
(Spectre / mainframes to
Meltdown) containers
ACI implements distributed network policies
• Contracts define Layer2-Layer4
security policies.
• ACI distributed security policies are
implemented at different
enforcement points:
- Leaf: hardware based, no External
L2/L3
performance penalty.
- vSwitch (i.e. OVS, AVE): closest
to VM, stateful connection vSwitch vSwitch w/OpFlex
tracking
You can combine both host-based and
network-based for tiered-security and
operational reasons
(SecOps vs. NetOps vs. DevOps).
Learn more: BRKACI-2010 - Tetration and ACI: Better Together
ACI RBAC
ACI Configuration Management
• ACI has 3 ways to manage the CLI WEB UI API
entire DC fabric; on and off prem
1. Web UI LDAP/AD AUTHORIZATION
RADIUS
2. CLI TACACS+
APIC-Local USER
3. API User to Roles/Domains
Mapping
Roles Domains
• All three methods are secured by What the user can do Where the user can do it
RBAC (Role Based Access ACI Anywhere
Control) and allow granular multi Remote Leaf / Virtual PoD APIC / Multi-Site Multi-Cloud Extensions
tenancy configuration for extra IP

WAN
IP
WAN
flexibility
• For example, if desired, your “Dev Remote Location On Premises Public Cloud
Tenant” admin cannot see or
Security Analytics Policy
configure your “Prod tenant” Everywhere Everywhere Everywhere
ACI RBAC Roles are very granular
ACI RBAC Roles are very granular
With the granular RBAC
capabilities there is a
capacity to split
roles/responsibilities for
users
Agenda
• Endpoint Identity using EPGs and Micro EPGs (uEPG)
• Improvements in Hardware Utilization
But before we talk about Micro EPGs
… a quick refresher on the ACI
policy model and endpoint domains
So What’s an Endpoint Group (EPG)?
Endpoints are “grouped” to attach them to the fabric.

An Endpoint Group (EPG) is a set of devices (or VMs)
that share the same policy requirements.
Communication Between EPGs
By default:
Endpoints inside an EPG can talk to each other
Endpoint Groups (EPGs) cannot communicate with each other.
To allow EPGs to speak with each other we connect them using contracts
ACI uses a “white list model.”
Remember…that’s the
default behavior. It can
be changed.
Now let’s talk about

contracts.
A typical 3-tier application with a non-virtualized
database
User traffic coming
WAN through the WAN
requires high
performance.
ALLOW ALLOW
ALLOW
TCP/443
TCP/3306 TCP/8080 Web Tier is VM
DB Tier is running
based. Load
in bare metal.
balancer required.
Database Tier App App App Web Web

DB DB VM VM VM
Web
(Bare Metal) VM VM VM
LB
App Server Tier Web Server Tier
(Virtual Machines) (Virtual Machines)
Contracts
WAN Contract
Contract ALLOW ALLOW

ALLOW Contract
TCP/3306 TCP/443
TCP/8080
Database Tier App App App Web

DB DB VM VM VM
Web Web
(Bare Metal) VM VM VM
LB
App Server Tier Web Server Tier
(Virtual Machines) (Virtual Machines)
uSegmentation
For Physical and Virtual A contract can use a Service Graph © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public
Understanding Fabric Domains
Fabric Domains represent the connectivity type for a physical port:
• Physical Domains (PhysDoms) – any type of endpoint (Bare Metal, IP
Storage, etc.)
• External Bridge Domains – endpoints that are on an external L2 fabric
• External Routed Domains – IP endpoints external to the fabric
• FibreChannel Domains – FC endpoints
• Virtual Network Domains (VMM Domains) – VM or Container endpoints
connected to the fabric via a domain manager that provides virtual
network information.
Understanding Fabric Domains (contd.)
• Fabric domains also help APIC manage the allocation of certain
resources for different types of endpoints:
• Encapsulation pools: VLAN and VXLAN
• Address pools: IP Multicast address pools
• Security Domains
• Ports associate to the Domains via Attachable Entity Profile (AEP)
Virtual Machine Manager (VMM) Domains for
Hypervisor Integration
 API Relationship is formed between APIC and
vCenter RHV-M Virtual Machine Manager (VMM)
API  APIC obtains virtualization inventory,
performs virtual and physical correlation
APIC has visibility of hypervisor hosts, Virtual

Machines, vNICs and more.
EXT
 APIC manages the hypervisor virtual switch
(VDS, OVS, AVE, etc …) via either API or
Opflex
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
 Multiple VMMs supported on a single ACI
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
Fabric simultaneously
VMM Domain 1 VMM Domain 2 VMM Domain 3 VMM Domain 4
Containers / PaaS Integration with ACI CNI Plugin
 Integration with kubeapi server, APIC
obtains inventory of nodes and Kubernetes
objects.
API
 ACI CNI Plugin Implements:
Distributed OVS Load Balancer for ClusterIP
services.
EXT Distributed HW Load Balancer for LoadBalancer

services.
Distributed Kubernetes Network Policies
OpFlex OVS OpFlex OVS  Secure multi-tenancy and ACI policies
 Visibility: Live statistics in APIC per

Node Node container, health metrics
Learn more: BRKACI-2505- Deploying Kubernetes in the Enterprise with Cisco ACI
BRKACI-3330 - Openshift and Cisco ACI© 2019
BRKACI-2301 Integration
Cisco and/or its affiliates. All rights reserved. Cisco Public 37
The ACI Network and Policy Model in one slide
Tenant – AcmeCo
VRF – MyVRF
Contracts
Bridge Domain – MyBD
Subnet/SVI
EPG – MyEPG EPG1 EPG2

Any-Any
Replicates a
Traditional Switch
L2 External EPG ≈ 802.1q Trunk

L3 External EPG ≈ L3 Routed Link
Three approaches to using EPGs in ACI
EPG/BD = VLAN EPG = App Tier Hybrid (Combination)
Create a BD and one EPG Create one EPG for each New Apps and Legacy
for each existing VLAN. application Tier. Apps share the same
Fabric.
Common strategy to lift- Flat-network design, many
and-shift traditional apps can share a single Tenant and VRF sharing.
configurations. BD.
or
Simpler for migration, Fantastic for GreenField
complex for Micro and automated Dedicated Tenant/VRF and
Segmentation. deployments. leaking.
A common network-centric example with
BD/EPG = VLAN
Tenant – AcmeCo
VRF – MyVRF
Bridge Domain – VLAN40 Bridge Domain – VLAN50 Bridge Domain – VLAN60

10.40.40.254/24 10.50.50.254/24 10.60.60.254/24
EPG – VLAN40 EPG – VLAN50 EPG – VLAN60
Micro EPGs allow the fabric
administrator to group endpoints based
on their attributes.
Understanding Micro EPGs Base EPG based on port
and encapsulation (i.e
VLAN or VXLAN)
• A MicroEPG (uEPG) is equivalent to a

EPG GREEN
regular EPG for all purposes, but
classification is based on endpoint
attributes (and dynamic in nature)
BM-01 BM-02 VM-01
• Endpoints assigned to the uEPG 10.10.10.11
f4:5c:89:b2:bf:cb
10.10.10.12
f4:5c:89:b2:ab:cd
10.10.10.13
regardless of the encapsulation/port

• The endpoint must be first known to a
regular EPG, called “base EPG”
Understanding Micro EPGs Base EPG based on port
and encapsulation (i.e
VLAN or VXLAN)

EPG GREEN
BM-01 BM-02 VM-01

f4:5c:89:b2:bf:cb
10.10.10.12
f4:5c:89:b2:ab:cd
10.10.10.13

• The endpoint must be first known to a uEPG MyDB
Define uEPG based on MAC.

Example:
Select MAC=f4:5c:89:b2:bf:cb
Understanding Micro EPGs
EPG GREEN
BM-02 VM-01
f4:5c:89:b2:ab:cd
10.10.10.13

uEPG MyDB uEPG Quarantine
• The endpoint must be first known to a
BM-01
10.10.10.11
f4:5c:89:b2:bf:cb
Define uEPG based on VM attributes.

Example:
VM-name=VM-01
The actual classification possibilities
depend on the type of endpoint Domain.
For endpoints connected to
Physical Domains (bare metal) you can
classify based on IP or MAC addresses.
Example: PhysDom (Bare Metal) with IP Address uEPG
IP Micro EPGs considerations on PhysDoms
• Base EPG must be configured and deployed to program VLANs on leaf
host ports
• Base EPG & IP uEPG must associate with same BD and the BD MUST
have an IP subnet configured.
• IP uEPG must be deployed an all the nodes where the BD is deployed by
using node attachment
• Deployment Immediacy must be “Immediate”
• You can specify individual IP addresses and/or subnets (i.e. 10.10.10.1,
10.10.10.0/24)
Software Dependency: 1.2(x)
Hardware Dependency: E-Series or newer
Caveat: bridged traffic will NOT be enforced based on the IP-EPG classification
PhysDom (Bare Metal) with IP Address uEPG
– Configuration [1/2]
1. Define uEPG and map to the same PhysDom and BD as Base EPG
2. Map the uEPG to the required leaf switches
PhysDom (Bare Metal) with IP Address uEPG
– Configuration [2/2]
3. Define the IP address and/or list of IP addresses to match on
4. Review the result
For endpoints connected to
VMware and Microsoft VMM Domains you can
use the IP, MAC or VM-attributes.
vSphere with VDS
Micro EPG Support with vSphere VDS
1. Start with Base EPG, enable MicroSeg
Must be Immediate!
Must be True!
EPG GREEN
ubuntu-01 centos-01 centos-02 ubuntu-02 APIC will then
configure the
dvPortGroup as an
isolated PVLAN
dvPortGroup GREEN (PVLAN p-3012, s-3019)

VMware VDS
GREEN GREEN
(v-3012/3019) (v-3012/3019)
Policy Enforcement
vSphere with VDS
1.1 Base EPG is working as normal EPG
EPG GREEN
ubuntu-01 centos-01 centos-02 ubuntu-02

VMware VDS
Communication
between endpoints
inside the EPG is
GREEN GREEN
allowed at the Leaf. (v-3012/3019) (v-3012/3019)
Proxy-ARP enabled.
Policy Enforcement
vSphere with VDS
2. Configure uEPG based on attributes
1. Define uEPG and map to the same VMM Domain and BD as Base EPG
Must be Immediate!
vSphere with VDS
2. Configure uEPG based on attributes
2. Configure the required attributes
We define attributes to match, in this

example, matching on the VM
Operating System (Ubuntu Linux)
vSphere with VDS
3. VM is classified according to attributes
uEPG UBUNTU EPG GREEN uEPG UBUNTU

ubuntu-01 centos-01 centos-02 ubuntu-02

VMware VDS VM name: ubuntu-01
IP: 0.0.0.0
MAC: 00:50:56:AD:15:2E
uEPG MAC Address
Ubuntu GREEN Ubuntu
Ubuntu GREEN
(v-3012/3019) (mac-list) (v-3012/3019) (mac-list)
VM name: ubuntu-02
vm-1 00:50:56:AD:15:2E
IP: 192.168.1.41
MAC: 00:50:56:AD:15:1F
vm-2 00:50:56:AD:15:1F
Policy Enforcement
A quick demo of using VM attributes for uSEG
Micro EPG Support with
vSphere VDS – Summary
Micro EPG Considerations on vSphere VDS
• Under base EPG you must enable micro segmentation for vDS. This is only required if
using uSeg with VDS.
• When EPG is mapped to VMM domain, it will change vDS and port-group
configuration: PVLAN will be enabled.
• Port-group uses secondary VLAN (isolated), which is same with intra-EPG isolation.
• Proxy-ARP is automatically enabled on base EPG (this is only supported in EX-
models)
• PVLAN configuration is only to force all traffic to flow through Leaf.
• Some considerations when using uEPG with VDS:

• Proxy-ARP is required
• SPAN/ERSPAN work at the leaf level, but operate on the Base EPG
• Traffic stats are kept for the Base EPG, not per uEPG
• Software Dependency: 1.3(1g)

• Hardware Dependency: EX-Series or newer
When using VMM Domains, you can
configure multiple attributes to select
endpoints for a Micro EPG using
Logical Operators
(since APIC release 2.3)
ACI Doing Segmentation With Attributes
uEPGs with Attributes and Logical Operators
- GUI Configuration (1/2)
Select “Match Any” for

‘OR’ Logic.
Click on ‘+’ to add additional

attributes to Match Any/All.
Or click ’+(‘ to add additional
sections.
Select “Match All” for
Select new “uSeg ‘AND’ Logic.
Attributes” folder under
each specific uEPG
uEPGs with Attributes and Logical Operators
- GUI Configuration (2/2)
Selects VMs with Tag ‘APP:OpenCart-Apache’, or VMs with

’Custom Attribute app-tier=app1-app’ as long as they are
running on vCenter DC1-EAST datacenter
What about Containers?
With Kubernetes Domains
classification does not use
micro EPG.
We use regular EPGs selected using
native semantics
(Kubernetes annotations).
Using annotations to specify the EPG for a set
of Kubernetes PODs
• Provide isolation beyond using Kubernetes Network Policies
• Opflex annotation can be applied at POD, Deployment or namespace
level
• Priority of annotations goes from less specific to more specific
1. Pod annotation
2. Deployment annotation
3. Namespace annotation
4. Namespace group mapping from controller config file
5. Global default group from controller config file
ACI allows flexible POD to EPG mapping K8s
Network
Policy
Cluster Isolation Namespace Isolation Deployment Isolation

Kube-default-EPG namespace-PROD-EPG Frontend-EPG API-Gateway-EPG
POD POD POD POD

POD POD Contract
POD POD POD POD
POD POD
POD POD POD POD
POD POD
Contract Contract Contract

POD POD namespace-QA-EPG Backend-EPG Monitoring-EPG
POD POD
POD POD
POD POD POD POD
POD POD POD Contract POD
POD POD POD POD
• Default behavior: single EPG for • Each namespace mapped to an • Each deployment mapped to an EPG
entire cluster user PODs EPG • Contracts control traffic between
• No need for internal contracts • Contracts for inter-namespace microservice tiers
traffic are required
Agenda
• Endpoint Identity using EPG and micro EPG (uEPG)
• Improvements in hardware utilization
By default communication between EPGs is not allowed
in absence of contracts
Without contracts, by
default there is no
communication
between groups
BM-01 VM-02 VM-03 BM-04

10.10.10.11 10.10.10.12 10.10.10.13 10.10.10.14
EPG BLUE EPG GREEN
Bridge Domain – 10.10.10.1/24
By default communication between EPGs is not allowed
in absence of contracts
any,tcp/8080
any, tcp/80
Contract: Blue-to-Green
Scope: VRF
CONSUMES
PROVIDES
Subject: AppTraffic
Both Directions: True
BM-01 VM-02 VM-03 BM-04
Reverse Port Filters: Yes
10.10.10.11 10.10.10.12 permit tcp/80 10.10.10.13 10.10.10.14
permit tcp/443
EPG BLUE EPG GREEN

BLUE Consumes the contract, GREEN Provides the contract,
so can connect to tcp/80 and so ports tcp/80 and tcp/443 are
tcp/443 exposed.
Demo: Creating a filter and contract
What about applying
policy within an EPG.
Denying all traffic between EPs in the same EPG
is easy
You can restrict all traffic inside a group using
Intra EPG isolation
• Supported on PhysDoms, VMware VMM domain using VDS (it also works
with AVS, AVE) (*)
• Since ACI 3.0 Microsoft VMM domain also supports intra EPG isolation.
• Can be configured on EPG and uEPG (**)
• It’s supported with EX and FX, and later, leaf models.
• We use Proxy-ARP – required to reach other EPG in the same subnet
• We utilize PVLAN integration already present for VDS intra EPG isolation.
Contracts can also apply for Intra-EPG
communications (since ACI 3.0)
• It is possible to assign contracts to
restrict traffic internal to an EPG
• It can be enabled on both EPG and uEPG
• It supported with PhysDoms and VMware
VMM Domains
• IntraEPG contracts require using proxy-
arp. This means it is only supported on
EX/FX switches or newer.
Intra EPG Security Feature Overview
How does it work for 2 VMs on the same ESXi Host?
VDS implementation:
1- We enable Intra-EPG Security TCP 22 allow
Deny all
Apply Intra-EPG Contract
2- APIC configures a PVLAN for the
EPG/portgroup on vCenter with the TOR APIC Proxy
setup as the promiscuous port Controller
ARP
3- The VMs in the portgroup can only talk to

the TOR Proxy ARP
ARP Request Response
4- When VM1 wants to talk to VM2 it ARPs for
it and the TOR responds with its MAC (proxy ARP) VDS
X
Enable
Private Web
5- All the traffic between the VMs is pulled to VLAN
vCenter
the TOR which allows only traffic permitted by VM VM
the contract applied for Intra-EPG security 1 2
IntraEPG Contract Use Case
– service vNIC used for mgmt in a clustered App
Example: a clustered web application. The jump host must be able to access all endpoints
and you cannot use IntraEPG Isolation because the required protocols must be allowed
between the VM inside the dvPortGroup.
EPG JumpHost
Contract: Zookeeper
10.90.90.15
Subject: Allow Zookeeper
TCP/2181
TCP/2888 Web-Tier PorGroup (BaseEPG)
TCP/3888
(PVLAN 2300/2301)
Web Web Contract: any-ip
intraEPG VM VM
Subject: Allow-any-ip
10.10.40.11 10.10.40.11 Any IP
web-prod-aci-01 web-prod-aci-02
app1-web
Only Zookeeper ports (uEPG)
allowed between VMs
Contracts also allow inserting services, like Next
Generation Firewalls, ADC, IPS/IDS, etc.
You can insert an NGFW, or Note: Service Graph between
a LB by attaching a Service Micro EPGs requires 3.2 or higher
Graph to the contract
subject
Contract: Blue-to-Green
Scope: VRF
Subject: AppTraffic
Both Directions: True
CONSUMES
PROVIDES
Reverse Port Filters: Yes
permit tcp/80
BM-01 VM-02 permit tcp/443 VM-03 BM-04
10.10.10.11 10.10.10.12 10.10.10.13 10.10.10.14
EPG BLUE EPG GREEN
vzAny allows you to configure contracts that
apply for all EPG in a VRF
Tenant VRF1
• vzAny represents the collection of EPGs
that belong to the same VRF, including L3 BD1
external.
EPG1
• Instead of associating contracts to each
individual EPG you can configure a contract
EPG2
to the vzAny
vzAny
• With cross-VRF contracts, vzAny can be a BD2
consumer, not provider EPG3
• Since ACI 3.2 it can also be used with
Service Graphs EPG4
Taboo Contracts
• Taboo are special type of contracts that will be applied to individual EPGs
• They deny a set of ports on the EPG to which the taboo contract is applied
• For instance you can say EPG Frontend does not allow tcp/445
• Taboo filters will override regular contract filters
I am not allowing any traffic
to TCP/445 regardless of
what other contracts say!
FRONTEND
BM-01 VM-02
10.10.10.11 10.10.10.12
Blacklist (Deny) Contracts – Introduced with ACI 3.2
• Representing policy with a whitelist model is complicated: it requires

complete understanding of application communication logic (every protocol
and port required).
• Since ACI 3.2 we are introducing the possibility of implementing blacklist
contracts.
• Blacklist contracts can apply to consumer/provider as well as intraEPG
configurations
• For instance, now you can:
• configure a contract that allows everything
• then deny specific ports and protocols.
Blacklist Contracts implement Deny Filter Rules
• Since 3.2, the subject filter attachment has one additional attribute action
• Action = permit, default to keep existing behavior
• Action = deny, causes rules for these filters to switch to deny (drop)
• Now subject filters will have different priorities to determine precedence:
• Default Values:
• Level 1 – lowest priority corresponding to any-to-any filter rules
• Level 2 – medium, corresponds to src-to-any/any-to-dst filter rules
• Level 3 – highest, corresponding to src-to-dst filter rules
Or to put it more simply…the most specific rule has highest priority
• Administrator is given a choice to override default priorities.
Simplifying Contract Configurations with
EPG Contract Inheritance
• Objective: simplify policy configuration
• Contract Inheritance will help:

- Establish a relation between EPGs, so that one EPG can inherit the contract relations of other
EPGs
- When new contract relations are added to the higher EPG, those with inheritance
relation will automatically get those same contract associations
• Considerations:
• does NOT apply to VzAny and also does NOT replace use of VzAny
• does NOT reduce number of contracts or TCAM utilization
• EPGs can only inherit contracts from EPGs under the same Tenant
Example: EPG_A has three contract relations
EPG_A Consumes Provides
Contract_DNS Contract_SSL
Contract_Internet
EPG_B is configured to inherit from EPG_A
Contract_Internet
EPG_B Consumes Provides

(Master: EPG_A)
Contract_Internet
Use the same contracts

as EPG_A
EPG_B is configured to inherit from EPG_A
- can now add specific contracts to “child”
Contract_Internet

(Master: EPG_A)
Contract_Internet Contract_TomCat
EPG_B also provides

another contract
EPG_C is configured to inherit from EPG_A
Contract_Internet

(Master: EPG_A)
EPG_C Consumes Provides EPG_C only gets

(Master: EPG_A) contracts from EPG_A
Contract_Internet
Changes to contract relations on EPG_A are
inherited by EPG_B and EPG_C

New contract relation added only
Contract_DNS Contract_SSL to EPG_A and automatically
Contract_Internet inherited by EPG_B and EPG_C
Contract_Ansible

(Master: EPG_A)
Contract_Ansible
EPG_C Consumes Provides

(Master: EPG_A)
Contract_Internet
Contract_Ansible
Adding Contract Inheritance
Contract Policy Enforcement can be enabled or
disabled at VRF level
• Policy Enforce: no communication without contracts
• Policy Unenforced: all communication allowed A binary configuration,
policy is either ON or OFF
for all EPG in the VRF
VRF – MyVRF VRF – MyVRF

L3Out L3Out
EPG-A EPG-B EPG-C EPG-A EPG-B EPG-C

External External
EPG EPG
Enabling Enforcement at VRF Level
Contract Preferred Group Example
Inside the
Preferred Group there VRF – MyVRF
is unrestricted
communication Preferred Group
EPG-A EPG-B EPG-C EPG-D L3Out

External
EPG
Excluded EPGs can

NOT communicate EPG-1
Contract-1
without contracts
EPG-3
EPG-2
Contract-2
Contract Preferred Group Example
VRF – MyVRF
Preferred Group
EPG-A EPG-B EPG-C EPG-D L3Out

External
EPG
Contracts are required

to reach EPG inside the
EPG-1 Preferred Group
Contract-3
Contract-1
EPG-3
EPG-2
Contract-2
Enabling Preferred Groups Is Easy
Contract Logging – Denied Packets
Logging Deny ACL deny not logged by default:
Fabric -> Fabric Policies -> Policies -> Monitoring -> Common Policy -> Syslog
• ACI can log implicit deny hits Message Policies -> Default -> Change ‘default’ to ‘info’
• For Bare Metal, VMware VDS and MSFT Domains
logs generated by Leaf
• For AVS logs may be generated on Leaf or vLeaf MySQLAccess
•
CONSUMES
For OpenStack ML2 mode, logs configured external to
PROVIDES
Subject: DB-Traffic
the fabric at the host Filter: Action:
• Syslog is exported according to monitoring policies icmp allow
VM-02 tcp/3106 allow VM-03
and configured External Data Collectors 10.10.50.101 10.10.10.200
• Logs include Tenant/VRF, EPG VLAN encap,
ingress interfaces and offending packet details SIP:10.10.50.101 DIP:10.10.10.200 Proto: 6 sPort:54135 dPort:125
• Software Dependency: supported on all software releases

• Hardware Dependency: supported on all hardware models
Feb 04 10:26:54 troy-leaf1 %LOG_LOCAL7-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_DENY:

CName: Test-Tenant:Test-Tenant-VRF(VXLAN: 2162689), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x00505690b43a,
DMac:0x0022bdf819ff, SIP: 10.10.50.101, DIP: 10.10.10.200, SPort: 54135, DPort: 125, Src Intf: port-channel2, Proto: 6, PktLen: 74
Contract Logging – Permitted Packets
Permit log configured at the
subject on a per filter basis.
Logging Permit
• Permit logging is configured per Filter
• For Bare Metal, VDS and MSFT Domains logs
generated by Leaf
• For AVS logs may be generated on Leaf or vLeaf MySQLAccess
• For OpenStack ML2 mode, logs configured external
CONSUMES
PROVIDES
to the fabric at the host Subject: DB-Traffic
Filter: Action:
• Syslog is exported according to monitoring icmp allow
VM-02 log
policies and configured External Data 10.10.50.101 tcp/3106 allow log
VM-03
10.10.10.200
Collectors
• Logs include Tenant/VRF, EPG VLAN
SIP:10.10.50.101 DIP:10.10.10.200 Proto: 1 sPort:0 dPort:0
encap, ingress interfaces and offending
packet details
• Software Dependency: 2.2(1n) or higher
• Hardware Dependency: requires EX models or newer
Feb 04 10:14:44 troy-leaf1 %LOG_LOCAL7-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-

ACLLOG_PKTLOG_PERMIT: CName: Test-Tenant:Test-Tenant-VRF(VXLAN: 2162689), VlanType: FD_VLAN, Vlan-Id: 21, SMac:
0x00505690b43a, DMac:0x0022bdf819ff, SIP: 10.10.50.101, DIP: 10.10.10.200, SPort: 0, DPort: 0, Src Intf: port-channel2, Proto: 1,
PktLen: 98
You can also see logs under the tenant
A quick work about
contracts in ACI Multi-
Site
ACI Multi-Site
Inter-Site Policies and ’Shadow Objects’
IP
Network
DP-ETEP A DP-ETEP B
 Inter-Site policies defined on the S2 S3 S4 S5 S6 S7 S8
S1
Multi-Site manager are pushed to
ACI Multi-Site
the respective APIC domains Orchestrator
 ‘Shadow Objects’ are created

(for non stretched objects) in EP1 EP2
each APIC domain to be able to Site 1 Site 2
enforce policy locally EP1
C
EP1
C EP2 EPG
EP2 EPG EPG
EPG
• Requires namespace translation function
performed by the spines Define and push inter-site policy
EP1 EP2
EPG
C EPG
Agenda
While for most use cases hardware
resource exhaustion for policy
enforcement is never going to be a
worry…you can have 128K policy
TCAM entries on a single FX*
switch.
We have added a few new features
for the very heavy policy users.
*Amount of TCAM depends on the switch type
Bi-Directional Compression added in ACI
Release 3.2
• Before 3.2 release, if the same contract was consumed and provided between
the same EPGs, that would result in 2 TCAM entries being programmed in HW.
One for consumer to provider and the other for provider to consumer
• After the introduction of the bi-directional compression feature in 3.2, if the

same contract is consumed and provided between two EPGs. It could be
represented in HW using a single TCAM entry with direction bit masked. This
would mean TCAM savings of 50%.
• When rules are compressed they lose per direction statistics. Given that, the
user has a choice to configure compression (TCAM savings) at the cost of
statistics.
Oversimplified Look TCAM Entries Before 3.2
Actual Contract Relationship Entry 1 EPG1

Any-Any
EPG2
Replicates a
Traditional Switch
Contracts
Entry 2 EPG1
Any-Any
EPG2
EPG1 EPG2 Replicates a
Traditional Switch
Any-Any
Replicates a
Traditional Switch
TCAM Entries After 3.2
Entry 1 EPG1
Any-Any
EPG2
Replicates a
Traditional Switch
Criteria for Bi-directional TCAM Compression
• Only the contracts which follow the below guidelines will become candidates of
bi-directional compression.
• Contract->subject->apply-both-direction
• Contract->subject->reverse-filter-ports
• Contract->subject->filter-group->no-stats
• Fully qualified rules
• Action: permit or permit+log
Policy Compression via Indirection In ACI 4.0
• All provider-consumer EPG pair refer to same set of rules in the policy CAM
• No statistics for these compressed rules
• This built-in HW support allows improving zonerule scale.
1st Stage: Policy 2nd Stage: Policy
Group Label Lookup Hash TCAM Lookup
Consumer sclass dclass PG-Label PG-Label Protocol sport dport
EPG2 EPG1 EPG2 10 10 tcp * 443
EPG1 EPG3 10
10 tcp * 80
Provider EPG1 EPG4 10
EPG1 Consumer
EPG3
One copy of contract rules for all consumer EPG

Consumer
EPG4
Policy compression requires FX and later hardware

Enabling Policy Compression
Filter
Directives available are:

Log and Enable Policy
Compression
Actions available are Permit

(default) and Deny
Contract Configuration Snapshot
Checking Filter/Contract Capacity on the
Dashboard
Operations->Capacity Dashboard->Leaf Capacity
Agenda
ACI Anywhere - Vision
Any Workload, Any Location, Any Cloud
ACI Anywhere
Remote Leaf / Virtual PoD APIC / Multi-Site Multi-Cloud Extensions
IP IP
WAN WAN
Remote Location On Premises Public Cloud
Security Everywhere Analytics Everywhere Policy Everywhere
Futures
ACI Hybrid-Cloud Deployment Model
ACI Multi-Site Hybrid Cloud:
Appliance
• AWS in Q2-CY19
• Azure in Q3-CY19
Site B
AWS Integration
Site A is in EFT with several customers
Site D
today
VM VM VM
VM VM VM
ACI – On Premise
VM VM VM
Common Discovery Policy Monitoring & Single Point Operational

Governance & Visibility Translation Troubleshooting Of Orchestration Consistency
Policy and Resource Mapping - AWS
User Account Tenant
Virtual Private Cloud VRF
VPC subnet BD Subnet
Tag / Label EP to EPG Mapping
Security Group EPG

Network Access List Taboo
Security Group Rule Contracts, Filters
Outbound rule Consumed contracts
Source/Destination: Subnet or IP or Any or ‘Internet’
Protocol
Port
Inbound rule Provided contracts
EC2 Instance
Network Adapter End Point (fvCEp)
Policy Mapping – AWS
Identity and Access
AAA Users, Security Domains
Management (IAM)
Region Pod
Availability Zone Path/Node Attachment

(AZ)
Infra
Overlay-1 VRF (ACI Infra)
VPC
VPC Peering Shared Services / Common
Internet Gateway,
VPN Gateway, Border Leaf, Spine (Internal and
External connectivity)
Direct Connect,
CSR1000V
Inter Region VPC Peering Inter POD Connectivity
Direct Connect Gateway
For your info
& reference
Policy Mapping - Azure

Resource Group Tenant
Virtual Network VRF
Subnet BD Subnet
Application Security Group EPG

(ASG)
Network Security Group

(NSG) Filters
Outbound rule Consumed contracts

Source/Destination: ASG or Subnet or IP or Any or ‘Internet’
Protocol
Port
Inbound rule Provided contracts

Virtual Machine
Network Adapter
End Point Learning on AWS
Multi-Site Orchestrator (MSO)
On-Premises Public Cloud
Site B • User deploys a new
Region 1
instance in AWS
• AWS config services
Infra VPC
notifies the event to cAPIC
AWS config
• cAPIC learns the endpoint
CSR CSR
services and registers it
• Based on the policies
Site A AZ-1 AZ-2
IPSec Tunnel
(EPG’s and Contracts) the
IPSec Tunnel VGW
correct security group
VGW (SG) is attached to the
instance
Security Group (SG) Availability Zone (AZ)

SG-1
CSR-1000V AWS Internet Gateway
User VPC - 1 User VPC -2 (IGW)
Cloud APIC AWS Virtual Private Gateway (VGW)
ACI Extensions to AWS
Multi-Site
Orchestrator
Site A On-Premises Public Cloud Site B
IP
Network
EPG EPG EPG
Contract Contract
Web APP DB
SG SG SG
SG Rule SG Rule
Web APP DB
AWS Region
VM VM VM

Extending ACI to the Cloud
ACI Multisite
Orchestrator
ACI for On-Premises
SG SG SG
SG Rule SG Rule
Web APP DB
IP
Network
EPG
Contract
EPG
Contract
EPG AWS Region
Web APP DB
Cloud ACI for Public Cloud
IP
Network
VM VM VM ASG ASG ASG
NSG NSG
Web APP DB
Azure Region

Cloud EPG
Mapping Endpoints by Tags WEB EPG DB EPG
Web-EPG associated
Site B •
to tag: “EPG: WEB”
• Web-EPG has
endpoints across
Us-East-1 & Us-West-
1 regions and multiple
subnets
Subnet-S1 – 10.1.1.0/24 Subnet-S3 – 10.1.3.0/24
• DB-EPG associated to
tag: “EPG:DB”
• DB-EPG has endpoints

across Us-East-1 &
Subnet-S2 – 10.1.2.0/24 Subnet-S4 – 10.1.4.0/24 Us-West-1 regions
and multiple subnets
US-East-1 US-West-1
Instances in a VPC and on-Premises
For traffic from Instances in a
On-Premises Public Cloud VPC to on-premises, traffic
Site B
Site A reaches CSR in Infra VPC
BGP EVPN
Control Plane AWS Region 1 and goes over the VXLAN
tunnel to the ACI spines on-
CSR CSR
premises
VXLAN TUNNEL
(DATA PLANE)
AZ-1 Infra VPC AZ-2 Spine forwards the traffic to

IPSec Tunnel
IPSec Tunnel
VGW
VGW
the corresponding leaf on
which the EP is located
EPG-1 EPG-1 EPG-2 EPG-3
EPG-1
VM SG-1 SG-1 SG-2 SG-3
Instance-1 Instance-2 Instance-3 Instance-4

User VPC - 1 User VPC -2
Creating an Application Using MSO
Traffic Allowed Between Two EPGs
Instances within a VPC
On-Premises Public Cloud
Site A Site B
AWS Region 1
All traffic between

CSR CSR instances within the same
VPC, can directly
AZ-1 Infra VPC AZ-2 communicate to each
IPSec Tunnel VGW
IPSec Tunnel other based on the
VGW
respective security group
Hybrid Web EPG Hybrid App EPG EPG-2 EPG-3
policies programmed by
cAPIC
SG-1 SG-1 SG-2 SG-3

Instances across VPCs
On-Premises Public Cloud For instances in two different
Site B VPCs communicating to each
Site A other, the traffic has to exit the
AWS Region 1
VPC either via VGW of the
user VPC and reach CSR in
CSR CSR infra VPC.
AZ-1 Infra VPC AZ-2

IPSec Tunnel IPSec Tunnel
VGW VGW Once the traffic reaches the
Hybrid-Web EPG Hybrid-App EPG CSR in infra VPC, packets are
EPG-1 EPG-1 EPG-2 EPG-3 routed to the destination based
on the configured policies
SG-1 SG-1 SG-2 SG-3

Demo
Demo
Tenant: CL-Demo
App: CL-Demo-App-1
EPG: CL-Base-EPG
10.99.1.10 10.99.1.11 10.99.2.10 10.99.2.11 10.99.3.10 10.99.3.11

CL-Base-EPG
Web Web App App DB DB
1 2 1 2 1 2
Web-App-DB-BD
10.99.1.1/24
10.99.2.1/24
10.99.3.1/24 Web App DB
Demo
Tenant: CL-Demo
App: CL-Demo-App-1
EPG: CL-Base-EPG
Classify by VM Name Classify by IP Address Classify by TAG
10.99.1.10 10.99.1.11 10.99.2.10 10.99.2.11 10.99.3.10 10.99.3.11
CL-Base-EPG
Web Web App App DB DB
1 2 1 2 3 2
Web-App-DB-BD
10.99.1.1/24
10.99.2.1/24
10.99.3.1/24 Web App DB
Summary
ACI enables micro segmentation that
you can deploy in a gradual and
flexible way. Use the right tool or
feature for your use case...
You can use static configurations
using the GUI or the NX-OS CLI …
… and you can use orchestration or
configuration management tools...
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
cs.co/ciscolivebot#BRKACI-2301
Complete your online
session survey
• Please complete your Online Session
Survey after each session
• Complete 4 Session Surveys & the Overall
Conference Survey (available from
Thursday) to receive your Cisco Live T-
shirt
• All surveys can be completed via the Cisco
Events Mobile App or the Communication
Stations
Don’t forget: Cisco Live sessions will be available for viewing

on demand after the event at ciscolive.cisco.com
Continue Your Education
Demos in Meet the Related

Walk-in
the Cisco engineer sessions
self-paced
Showcase labs 1:1
meetings
Thank you

Brkaci 2301

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Brkaci 2301

Uploaded by

Copyright:

Available Formats

BRKACI-2301

Practical Applications of Cisco

• Perimeter security is not enough: once

172.16.10.11 172.16.10.12 172.16.10.13

172.16.10.11 172.16.10.12 172.16.10.13

Web Servers Very simple

172.16.10.11 172.16.10.12 172.16.10.13 172.16.10.14 172.16.10.15 172.16.10.16

172.16.10.11 172.16.10.12 172.16.10.13 172.16.10.14 172.16.10.15 172.16.10.16

Virtualized w/ VMware Bare Metal / Big Data Shared/Infra

Endpoint Identity Policy Definition Verify and Refine

Classify endpoints into Determine what policy to Verify policy enforcement,

- Network identity - Application Dependency - Policy visibility

Network-based - Centrally manage access rules at the network edge (Virtual

tenancy configuration for extra IP

Endpoints are “grouped” to attach them to the fabric.

Endpoints inside an EPG can talk to each other

Endpoint Groups (EPGs) cannot communicate with each other.

ACI uses a “white list model.”

Now let’s talk about

Database Tier App App App Web Web

Contract ALLOW ALLOW

Database Tier App App App Web

APIC has visibility of hypervisor hosts, Virtual

EXT Distributed HW Load Balancer for LoadBalancer

Distributed Kubernetes Network Policies

OpFlex OVS OpFlex OVS  Secure multi-tenancy and ACI policies

 Visibility: Live statistics in APIC per

EPG – MyEPG EPG1 EPG2

L2 External EPG ≈ 802.1q Trunk

EPG/BD = VLAN EPG = App Tier Hybrid (Combination)

Bridge Domain – VLAN40 Bridge Domain – VLAN50 Bridge Domain – VLAN60

EPG – VLAN40 EPG – VLAN50 EPG – VLAN60

• A MicroEPG (uEPG) is equivalent to a

regardless of the encapsulation/port

• A MicroEPG (uEPG) is equivalent to a

• Endpoints assigned to the uEPG 10.10.10.11

regardless of the encapsulation/port

Define uEPG based on MAC.

regardless of the encapsulation/port

Define uEPG based on VM attributes.

2. Map the uEPG to the required leaf switches

4. Review the result

dvPortGroup GREEN (PVLAN p-3012, s-3019)

dvPortGroup GREEN (PVLAN p-3012, s-3019)

We define attributes to match, in this

uEPG UBUNTU EPG GREEN uEPG UBUNTU

dvPortGroup GREEN (PVLAN p-3012, s-3019)

• Some considerations when using uEPG with VDS:

• Software Dependency: 1.3(1g)

Select “Match Any” for

Click on ‘+’ to add additional

Selects VMs with Tag ‘APP:OpenCart-Apache’, or VMs with

Cluster Isolation Namespace Isolation Deployment Isolation

POD POD POD POD

Contract Contract Contract

BM-01 VM-02 VM-03 BM-04

EPG BLUE EPG GREEN

Bridge Domain – 10.10.10.1/24

Bridge Domain – 10.10.10.1/24

3- The VMs in the portgroup can only talk to

Web Web Contract: any-ip

EPG BLUE EPG GREEN

Bridge Domain – 10.10.10.1/24