ACI_Feature_Updates

ACI 6.0.
3F Version Update
ACI PIW
Anirudh Kashyap, Technical Marketing Engineer
ACI PIW
November 23, 2023
▪ Summary of enhancements
▪ New Hardware Feature
Agenda ▪ Architecture Enhancements
▪ Serviceability Enhancements
▪ GUI Walkthrough
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
Summary of Enhancements
Architecture Simplicity/Serviceability Scaling

• AzureStack HCI Approved Vendor List • Hitless SMU for OSPF • 2K VRFs per Leaf
• Nutanix VMM Domain • Rogue EP Control Enhancements • Multi-tenant scaling
• vzAny PBR and L3Out PBR for Multisite • Transaction based logging enhancements
• Stretch L3out SVI across RL pairs • Memory based switch upgrade
• NSX-T Integration with Policy Mode
• TCP Adjust MSS for Mpod, Msite, RL
New Hardware Feature
All GX2 Switches support the 100G to 25Gx4 breakout
9364D
Current GX2 Switches are
• N9K-C9364D-GX2A
• N9K-C9332D-GX2B
• N9K-C9348D-GX2A 9348D
• N9K-C9408
100G to 25Gx4 Breakout is supported only on access ports for

the above switches in leaf mode only 9332D
9300GX2 Switches
9400GX2 Switches
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 9408 Switch
vzAny and L3Out PBR for Multisite
Current Limitations
• East-West (EPG-to-EPG) intra-VRF and inter-VRF contract with PBR, PBR enforcement pin down to
provider side. Consumer EPG is network centric and no support for host subnet.
• North-South (L3Out-to-EPG) intra-VRF and inter-VRF contract with PBR, PBR enforment is on EPG
(NBL) side
• In the case of inter-VRF, the L3Out EPG must be the provider
• EPG-->L3out direction, no pctag translation
• L3Out-L3Out intra-VRF and inter-VRF contract with PBR is NOT supported
• vzAny-to-EPG/L3Out or vzAny-to-vzAny contract with PBR is NOT supported.
Use-cases and issues
New Use-cases and Requirements from Customers
• Intra-vrf vzAny-vzAny
• Intra-vrf vzAny-EPG
• Intra-vrf vzAny-l3Out
• Intra/inter vrf l3Out-l3Out
• Support both network centric and application centric mode
Major Issues addressed
• Achieve Symmetricity for different traffic combination
• Overcome the invalid sclass for intersite traffic to l3out
• Support app-centric mode - No traffic drop
• Avoid continuous traffic Tromboning
• Network-centric mode - Host route issue
Feature overview
• Vzany-vzany, vzany-l3out, l3out-l3out - Traffic goes to both site FWs with aclRule
• Vzany-EPG - PBR is enforced on the EPG side using SPI/DPI flag
• Use Site info as filter - To solve the app-centric first packet drop issue using sg_label
• Conversational learning - To avoid continuous traffic tromboning
• Multisite tracking - PBR policy can redirect to remote site FWs
NDO GUI
ACI GUI
ACI VMM Integration with Nutanix
Nutanix Cluster
AOS Leader AOS Worker AOS Worker

CVM CVM CVM
PE Leader PE Worker PE Worker
VM1 Host1 VM2 VM1 Host2 VM2 VM1 Host3 PC
AHV OVS AHV OVS AHV OVS
Pre-requisites and considerations of Nutanix Integration
• The Nutanix Cluster should be setup correctly

• AHV Cluster should be registered with the Prism Central
• Cluster should be running with an AOS version >= 6.5
• Prism Central Management should be UP and available
• Prism Element should be UP and available
• Enable micro segmentation on Prism Central which will be needed for intra EPG
isolation
• The custom switch name is optional. If not specified vmmDomain creation will create
a vswitch on Nutanix with the domain name
• If cluster AOS version is 6.5, customSwitchName field is mandatory. For AOS version
6.6, customSwitchName is optional.
Software Architecture of Nutanix Integration
Nginx PolicyDist PolicyMgr Doer Event Listener
VmmMgr
Nutanix Controller Event Collector
PC/PE
Once the vmmDomain is created on APIC. The following object will be

created on the Nutanix side
• New vswitch with domain name, if custom switch name was not
specified
• Webhooks for event notification
UI:Configure - vmmDom
UI:Configure – DHCP Pool
UI:Configure – EPG
UI:Inventory
UI:Statistics
LLDP for Azure stack HCI
Requirements for AzureStack HCI 20H2
Requirements for AzureStack HCI 22H2
UI Configuration breadcrumbs
LLDP default policy:
LLDP default policy under Fabric-> Policies -> Global -> LLDP Policy default and
select lldp optional TLVs in the Optional TLV Selector part.
LLDP interface policy:

LLDP Interface policy to select the DCBX version under Fabric -> Access Policies -
>Policies ->Interface -> LLDP interface -> default -> DCBX version: CEE or IEEE
802.1
Configuration guide:
https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_APIC_CDP_and
_LLDP_Management_Interface.html
NSX-T Integration with Policy Mode
Understanding Policy Mode • NSX-T release 2.4 introduced a new policy API.
• VMware announced the deprecation of Manager
mode APIs and UIs.
• It uses a declarative API model and can be used to
create the entire intent in one go without caring about
ordering or having to make multiple API calls.
• This reduces the number of configuration steps
drastically.
• NSX-T Policy API has a simplified data model and
can be consumed with a now easier, intent-based
approach.
• From NSX-T 2.4 release, users interact with the NSX
Manager using the Simplified UI.
• The traditional objects will be available under the
Advanced UI.
Changed Object Mapping in APIC
Management API Policy API ACI equivalent
Logical switch segment Port group
T1 logical router Tier 1 Gateway NA
T0 logical router Tier 0 Gateway NA
NSgroups, IP sets, MAC sets Group NA
Firewall section Security policy NA

Firewall rule Rule NA
Edge firewall Gateway firewall NA
UI Config First time EPG Creation
UI Configuration – While adding VMM Domain after
EPG Creation
Migrate from Management API to Policy API
NSX-T ACI
TCP MSS
About TCP MSS
• ACI fabric adds an iVxlan header to packets.

• It adds 54 bytes to original packet.
• IPN between ACI switches may consist of third-party
switches/routers whose MTU may not support the extra 54 bytes.
• TCP MSS sent by host during session establishment does not
account for the extra 54 bytes, this leads to drop.
TCP MSS Feature detail
• For TCP sessions, TCP-SYN packets carry Maximum Segment Size (MSS).
• MSS informs hosts to send TCP packets adhering to this size.
• MSS = MTU - IP Hdr Size – TCP Hdr Size
• ACI fabric can intercept these packets and reduce the MSS to a customer
defined threshold such that packet size does not exceed IPN/WAN MTU.
• Works only for TCP sessions.
• Not developed for UDP packets at this time.
TCP MSS Adjust Deployment modes
• Location aware(RL/MPOD/MSITE)
• ToRs with same TEP-POOL are considered in same location. ()
• No TCP-MSS adjust if the src/dst tors are in same tep-pool.
• Do TCP-MSS adjust if the src/dst tors are NOT in same tep-pool
• Fabric level knob
• SUP Punt is on on egress ToR
• Location Unaware(only For RL)

• Do TCP-MSS adjust for all session starting on RL.
• Exception: intra leaf, VPC peer
• SUP Punt is on ingress RL
UI Configuration of TCP MSS Adjust
OSPF Hitless SMU
• Before the implementation of feature Hitless SMU, OSPFv2 and OSPFv3 processes are not restart
able. This implies that in case OSPFv2 or OSPFv3 process crashed/restarted the box would reload
• Restart SMU for OSPF would be supported from 6.0.3, earlier only reload SMU was supported for
OSPF
CLI Implementation
• OSPF process should be able restart when issuing a CLI command from ibash.
• kill -9 `pidof ospf`

• kill -9 `pidof ospfv3’
• This command will trigger system manager to request for a cleanup from OSPFv2/v3 and eventually
exit. Post exit of process, system manager will start the process again treating the exit as graceful
process exit.
Rouge EP Exception List – Feature Overview
Rogue EP:
• Developed to prevent rapidly moving MAC/IP Eps.
• Rogue Eps are marked static temporarily.
• Rogue fault is raised so that corrective action can be taken by the user.
Rogue Exception list:

• Movement of certain endpoints is desirable in some scenarios in which rogue restrictions caused
undesired deployments.
• To address these requirement came to have rogue exception list for some Eps
• Rogue restrictions can be relaxed for such endpoints based on the user needs.
Rogue Exception Wildcard :

• Exhausting for users to enter each MAC EP under BD one by one.
• Customers like Fedex asked for wild card support. Two kind of wild cards will be supported
• Rogue exception list wild card entry of {*, MAC} at fabric level.
• Rogue exception list wild card entry of {BD, *} at BD level.
Rouge EP Exception List Feature Highlights
• List of MAC addresses which can be exempted from default global Rogue behavior.
• MAC addresses in the exception list will use a higher move threshold.
• When EP moves more than 3000 times in a 10-minute interval, It will be marked as
static/rogue for 30 seconds.
• Rogue exception behavior applies only to the MAC addresses and not for any IP address
associated with the MAC.
• MACs under this list are also registered for relaxed coop dampening on the spine.
GUI Config for BD
GUI config for wild-card MAC
GUI Config for L3Out
Q&A

ACI_Feature_Updates

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ACI_Feature_Updates

Uploaded by

Copyright:

Available Formats

ACI 6.0.

▪ New Hardware Feature

Agenda ▪ Architecture Enhancements

Architecture Simplicity/Serviceability Scaling

100G to 25Gx4 Breakout is supported only on access ports for

• Intra/inter vrf l3Out-l3Out

• Support both network centric and application centric mode

Major Issues addressed

• Achieve Symmetricity for different traffic combination

• Overcome the invalid sclass for intersite traffic to l3out

• Support app-centric mode - No traffic drop

• Avoid continuous traffic Tromboning

• Network-centric mode - Host route issue

• Vzany-EPG - PBR is enforced on the EPG side using SPI/DPI flag

• Conversational learning - To avoid continuous traffic tromboning

• Multisite tracking - PBR policy can redirect to remote site FWs

AOS Leader AOS Worker AOS Worker

VM1 Host1 VM2 VM1 Host2 VM2 VM1 Host3 PC

AHV OVS AHV OVS AHV OVS

• The Nutanix Cluster should be setup correctly

Nutanix Controller Event Collector

Once the vmmDomain is created on APIC. The following object will be

Requirements for AzureStack HCI 22H2

LLDP interface policy:

Firewall section Security policy NA

• ACI fabric adds an iVxlan header to packets.

• Location Unaware(only For RL)

• kill -9 `pidof ospf`

Rogue Exception list:

Rogue Exception Wildcard :

You might also like