BRKDCN 1645

Introduction to VXLAN
The future path of your

datacenter
Rahul Parameswaran, Technical Marketing Engineer
BRKDCN-1645
Cisco Webex Teams
Questions?
Use Cisco Webex Teams to chat
with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
A few things..
• Prerequisites
• Good understanding of Unicast Routing Protocols – OSPF/ISIS
• Knowledge of Multi protocol BGP (MP-BGP)
• Basics of Multicast forwarding and PIM
• Use Cisco WebEx Teams for Questions
• Watch out for the hidden slides ☺
Session Objective
• A short overview on Data Center Evolution

• Introduction to Overlays and VXLAN
• Understanding how MP-BGP is used as a control plane
• Packet Walk with VXLAN
• Design options and additional use cases
Agenda
▪ Data Center evolution

▪ Overlay Taxonomy
▪ VXLAN with MP-BGP EVPN Control Plane
▪ Packet Walk
▪ VXLAN Design Options
▪ Use cases
Data Center “Fabric” Journey
Layer-3 HSRP HSRP
Layer-2
Spanning-Tree
Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2
Baremet al Hypervisor Hypervisor Hypervisor Baremet al Hypervisor Baremet al Baremet al Hypervisor Hypervisor
Spine Spine Spine Spine
ACI
VPC VPC
Layer 3 VTEP VTEP VTEP VTEP VTEP VTEP VTEP
Layer 2
Baremetal Hypervisor Hypervisor Hypervisor Baremetal Hypervisor Baremetal Baremetal Hypervisor Hypervisor
Why VXLAN Overlay
Customer Needs VXLAN Delivered

Any workload anywhere – VLANs limited Any Workload anywhere- across Layer
by L3 boundaries 3 boundaries
VM Mobility Seamless VM Mobility

Scale above 4k Segments (VLAN
Scale up to 16M segments
limitation)
Leverages ECMP for optimal path usage
Efficient use of bandwidth
over the transport network
Secure Multi-tenancy Traffic & Address Isolation
Overlay Taxonomy
Identifier = VN Identifier (VNID) Overlay Control Plane

VTEPs
NVE = Network Virtualisation Edge
VTEP = VXLAN Tunnel End-Point
Encapsulation
Edge Devices (NVE)

Edge Device (NVE)
Hosts
Underlay Network (end-points,
physical or
virtual)
Underlay Control Plane
VXLAN Packet
• VXLAN is point to multi-point tunneling mechanism to extend Layer 2 networks over an IP network
VXLAN Tunnel
Ethernet Frames
Host IP Network Host
NETWORK
OVERLAY
1 4
Host Host
2
Switch 1 Switch 2 5
Host Host
3 6
IP/UDP Packets
Host
7
Host
Switch 3 8
Host
9
• VXLAN uses MAC in UDP encapsulation (UDP destination port 4789)
Outer Outer Outer Outer Outer Outer VXLAN Inner Inner Optiona Original CRC
PLANE
DATA
MAC MAC 802.1Q IP DA IP SA UDP ID MAC MAC l Inner Ethernet

CRC
DA SA (24 bits) DA SA 802.1Q Payload
VXLAN Encapsulation Original Ethernet Frame
Lets Build a
VXLAN Fabric
VXLAN Fabric – Creating the underlay network
IP routed Network
• Flexible topologies
• Recommend a network with redundant paths using ECMP for load sharing
• Support any routing protocols --- OSFP, IS-IS, BGP, etc.
• All proven best practices for IP routing network apply
AnyCast RP ip pim rp-address 10.237.1.1 group-list 224.0.0.0/4

ip pim anycast-rp 10.237.1.1 10.255.255.101
ip pim anycast-rp 10.237.1.1 10.255.255.102
ip pim rp-address 10.237.1.1 group-list

224.0.0.0/4 Spine router ospf 1
interface Ethernet1/50
interface Ethernet1/49 mtu 9216
ip address 192.168.1.0/31 ip address 192.168.1.10/31
mtu 9216 ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
Leaf
Two Modes of VXLAN
Flood-and-Learn VXLAN: VXLAN EVPN:

• No control plane • EVPN as control plane
• Data driven flood and learning • VTEPs exchange L2/L3 host and
→ Ethernet in the overlay network subnet reachability through EVPN
control plane
→ Routing protocol for both L2 and L3
forwarding
• Limited scale • Increased scale and stability

• Limited workload mobility • Optimized workload mobility
• Centralized Gateway • Distributed Anycast Gateway
• Security Risk • Increased Security
VXLAN BUM Traffic Handling
• BUM Traffic --- Multi-destination traffic
• Broadcast
• Unknown Layer-2 Unicast
• Multicast
BUM Traffic transport mechanisms

• Multicast replication
Requests the underlay network to run IP multicast
• Ingress unicast replication
One unicast replica per remote VTEP
Increase traffic load throughout the network
VXLAN with BGP
EVPN Control
Plane
EVPN Primer --- MP-BGP Review
Virtual Routing and Forwarding (VRF)
Layer-3 segmentation for tenants’ routing
space
BGP advertisement:
VPN-IPv4 Addr = RD:16.1/16
Route Distinguisher (RD): BGP Next-Hop = PE1
Route Target = 100:1
8-byte field, VRF parameters; unique value to eBGP: eBGP:
make VPN IP routes unique: RD + VPN IP prefix

16.1/16 16.1/16
IP Subnet IP Subnet
PE1 P P PE2
Selective distribute VPN routes:
CE1 Blue VPN
Route Target (RT): 8-byte field, VRF parameter,

unique value to define the import/export rules ip vrf
VRF
Name
blue-vpn
parameters:
RD 1:100
= blue-vpn
for VPNv4 routes RDroute-target

= 1:100 export 1:100
route-target
Import import =
Route-Target 1:100
100:1
Export Route-Target = 100:1
VPN Address-Family:
Distribute the MP-BGP VPN routes
What is VXLAN/EVPN?
• Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)

• Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
• Forwarding decision based on Control-Plane (minimizes flooding)
• Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
Control-
EVPN MP-BGP - RFC 7432
Plane
Provider Backbone Bridges

Data- Multi-Protocol Label Switching (MPLS) Network Virtualization Overlay (NVO)
(PBB)
Plane draft-ietf-l2vpn-evpn draft-sd-l2vpn-evpn-overlay
draft-ietf-l2vpn-pbb-evpn
➢ EVPN over NVO Tunnels (VXLAN, NVGRE, MPLSoE) for Data Center Fabric encapsulations
➢ Provides Layer-2 and Layer-3 Overlays over simple IP Networks
Layer 2 Multi-tenancy
Switch level multi-tenancy

• VLAN to Segment ID mapping
(4K vlans per switch)
• With VLAN we can achieve
per port significance
Layer 3 Multi-tenancy
Spine
Tenants or VRF for L3 logical separation

VRF VRF VRF
VTEP VTEP VTEP VTEP

Leaf
EVPN based VXLAN Fabric RR EVPN Route Reflector
RP Rendezvous Point (Underlay)
! spine bgp config
router bgp 65001

router-id 10.1.0.5
RP RR RP neighbor 10.1.0.1
RR
remote-as 65001
update-source loopback0
! leaf bgp config address-family l2vpn evpn
router bgp 65001 send-community
router-id 10.1.0.4 Spine send-community extended
neighbor 10.1.0.5 route-reflector-client
remote-as 65001
address-family l2vpn evpn
MP-iBGP Sessions
VXLAN/EVPN Fabric
send-community
send-community extended VRF VRF
vrf VRF-RED
address-family ipv4 unicast
advertise l2vpn evpn
Leaf
VTEP VTEP VTEP VTEP VTEP
vrf VRF-BLUE
Service Leaf Border Leaf
Compute Leaf
EVPN Control
MP-BGP Plane
for VXLAN EVPN– Reachability
Control Plane Distribution
• EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC Spine
• Host-IP
• Internal IP Subnet
• External Prefixes
VTEP VTEP VTEP VTEP Leaf
▪ Use MP-BGP with EVPN Address Family on leaf nodes to distribute

internal host MAC/IP addresses, subnet routes and external reachability
information
▪ MP-BGP enhancements to carry up to 100s of thousands of routes with
reduced convergence time
Configuration Snippet
Vlan 10
vn-segment 5010
Vlan 20 Layer 2 VNI Spine
vn-segment 5020
Vlan 1000
!Layer 3 VNI Layer 3 VNI VRF VRF
vn-segment 9999
Vlan 2000
!Layer 3 VNI VTEP VTEP VTEP VTEP Leaf
vn-segment 9998
interface Vlan10 Host 1 Host 2 Host 3

no shutdown H-MAC-1 H-MAC-2 H-MAC-3
vrf member VRF-RED H-IP-1 H-IP-2 H-IP-3
ip address 192.168.10.254/24 tag 12345 VLAN 10 VLAN 20 VLAN 10
ipv6 address 2001::1/64 tag 12345 VXLAN 5010 VXLAN 5020 VXLAN 5010
fabric forwarding mode anycast-gateway
interface Vlan20
no shutdown Layer 3 VNI
vrf member VRF-BLUE vrf context VRF-RED vrf context VRF-BLUE
ip address 192.168.20.254/24 tag 12345 vni 9999 vni 9998
ipv6 address 2002::1/64 tag 12345 rd auto rd auto
address-family ipv4 unicast address-family ipv4 unicast
interface nve1 Map L2VNI to route-target both auto route-target both auto
source-interface loopback0 NVE route-target both auto evpn route-target both auto evpn
host-reachability protocol bgp evpn evpn
member vni 5010 vni 5010 l2 vni 5020 l2
mcast-group 239.1.1.1 rd auto rd auto
member vni 5020 route-target both auto route-target both auto
mcast-group 239.1.1.1
member vni 9999 associate-vrf
member vni 9998 associate-vrf Associate L3VNI
to NVE
Distributed Anycast Gateway in MP-BGP EVPN
# VLAN to VNI mapping

vlan 20
vn-segment 5020
# Anycast Gateway MAC, identically configured on all VTEPs

fabric forwarding anycast-gateway-mac 0002.0002.0002
# Distributed IP Anycast Gateway (SVI)

The same anycast gateway virtual # Gateway IP address needs to be identically configured on all
IP address and MAC address are VTEPs
configured on all VTEPs in the VNI. interface vlan 20
no shutdown
vrf member VRF-BLUE
ip address 192.168.20.254/24
ipv6 address 2002::1/64
SVI SVI SVI SVI
GW IP GW IP GW IP GW IP
GW MAC GW MAC GW MAC GW MAC
VTEP VTEP VTEP VTEP
Host 1 Host 2 Host 3 Host 4

MAC1 MAC2 MAC3 MAC4
IP 1 IP 2 IP 3 IP 4
VLAN A VLAN A VLAN A VLAN A
VXLAN A VXLAN A VXLAN A VXLAN A
EVPN Peer and Endpoint(Host) Discovery
Triggered by Host Communication across the same VLAN/VNI (L2)
End System End System Overlay Forwarding Table
Host1 <MAC-A> , VTEP 1, L2-VNI
S-MAC: MAC-1
BGP EVPN Type-2 MAC
D-MAC: 3 ARP Request for IP B
01:00:5E:01:01:01
update Src MAC: MAC-A
Advertises MAC-A with Outer S-IP: IP-1 Dst MAC: FF:FF:FF:FF:FF:FF
MAC-3
IP-3
VTEP 3
VTEP-
NH:IP-1 (VTEP-1) L2VNI Outer D-IP: 239.1.1.1
3
UDP
VXLAN VNID: 5010
ARP Request for IP B 4
Src MAC: MAC-A
Dst MAC: ARP Response from IP
7 FF:FF:FF:FF:FF:FF
2
B
2 Multicast Group
VTEP 2
IP-2
Src MAC: MAC-B
Dst MAC: MAC-A End System B
ARP Response from IP MAC-B
MAC-2
B 239.1.1.1 S-MAC: MAC-2
Src MAC: MAC-B VTEP-1 2 D-MAC: MAC-1 VTEP-2 3 IP-B
Dst MAC: MAC-A 5 Outer S-IP: IP-2 ARP Request for IP B

End System A 1 VTEP 1
IP-1 6
Outer D-IP: IP-1 Src MAC: MAC-A
Dst MAC: FF:FF:FF:FF:FF:FF
MAC-A
MAC-1 UDP
IP-A
ARP Request for IP B
Src MAC: MAC-A VXLAN VNID: 5010 Overlay Forwarding Table
ARP Response
Host1 <MAC-A> , VTEP 1, L2-VNI
from IP B
Src MAC: MAC-B
Dst MAC: MAC-
A
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
EVPN Peer and Endpoint(Host) Discovery
Triggered by Host Communication between VLAN/VNI (L3)
End System End System
BGP EVPN Type-2 MAC +Host IP update

Advertises MAC-A, IP-A with NH:IP-1
(VTEP-1) Router MAC:MAC-1 L2VNI, L3VNI
MAC-3
IP-3
VTEP 3
Overlay Forwarding Table
VTEP-
Host1 <IP-A> , VTEP 1, L3-VNI
3
VTEP 2
2 IP-2
2 Multicast Group
MAC-2
End System B
ARP Response from MAC-B
VTEP1
VTEP-1
239.1.1.1 IP-B
Src MAC: GW-MAC VTEP-2
Dst MAC: MAC-A
End System A 1 VTEP 1
IP-1
MAC-A
IP-A MAC-1
ARP Request for anycast
GW at VTEP1 Overlay Forwarding Table
Src MAC: MAC-A
Host1 <IP-A> , VTEP 1, L3-VNI
Src IP : IP-A
Packet Walk
Communication between hosts in same VLAN/VNI
Outer S-MAC: MAC-3
Outer S-MAC: MAC-1 Outer D-MAC: MAC-4
Outer D-MAC: MAC-2
3 Outer S-IP: IP-1
Outer S-IP: IP-1 Outer D-IP: IP-4
Outer D-IP: IP-4 Routed Based on
UDP
UDP Outer IP header
VXLAN VNID: 5010
VXLAN VNID: 5010 (L2 VNI)
IP Network S-MAC: MAC-A
S-MAC: MAC-A Underlay Underlay D-MAC: MAC-B
D-MAC: MAC-B Router-1 Router-2 S-IP: IP-A
S-IP: IP-A MAC-2 MAC-3 D-IP: IP-B
D-IP: IP-B 2 IP-2: IP-3: 4
165.123.1.2 140.123.1.2 MAC-4
IP-4:
MAC-1 140.123.1.1
S-MAC: MAC-A IP-1:
D-MAC: MAC-B
VTEP-1 S-MAC: MAC-A VTEP-2
165.123.1.1 D-MAC: MAC-B
S-IP: IP-A
D-IP: IP-B 1 S-IP: IP-A 5
D-IP: IP-B
Host-B
Host-A
MAC-A MAC-B
VXLAN VNID 5010
IP-A: IP-B:
192.168.10.1 192.168.10.20
0 Vlan 10
Vlan 10 VNI 5010
VNI 5010
Packet Walk
Communication between hosts in different VLAN/VNI
Outer S-MAC: MAC-3
Outer S-MAC: MAC-1 Outer D-MAC: MAC-4
Outer D-MAC: MAC-2
3 Outer S-IP: IP-1
Outer S-IP: IP-1 Outer D-IP: IP-4
Outer D-IP: IP-4 Routed Based on
UDP
UDP Outer IP header
VXLAN VNID: 9999
VXLAN VNID: 9999 (L3
VNI) Underlay IP Network Underlay S-MAC: MAC-1
S-MAC: MAC-1 Router- Router-
D-MAC: MAC-4
D-MAC: MAC-4
1 MAC-2 MAC-3 2
S-IP: IP-A
D-IP: IP-B
S-IP: IP-A
D-IP: IP-B 2 IP-2: IP-3: 4
165.123.1.2 140.123.1.2
VTEP-1
S-MAC: MAC-A VTEP-2
D-MAC: GW-MAC (L3 GW) S-MAC: MAC-4
MAC-1 D-MAC: MAC-B
(L3 GW)
S-IP: IP-A
MAC-4
D-IP: IP-B 1 IP-1:
S-IP: IP-A 5 IP-4:
165.123.1.1 D-IP: IP-B 140.123.1.1
Host-B
Host-A
MAC-A MAC-B
VXLAN L3 VNID 9999 (Tenant VRF A)
IP-A: IP-B:
192.168.10.10 192.168.20.10
Vlan 10, Vlan 20,
VNI 5010 VNI 5020
EVPN Control
VXLAN Plane ---
BGP Control VM Mobility
Plane
NLRI: Spine
• Host H-MAC-1, H-IP-1
• NVE VTEP-1
• VNI 5000
Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf
• Cost
• Sequence number :0 Host 1
H-MAC-1
H-IP-1
VLAN 10
VXLAN 5000
MAC IP VNI Next-Hop Encap Seq#
1. Host 1 attaches to VTEP-1
H-MAC- H-IP-1 5000 VTEP-1 VXLAN 0
1
2. VTEP-1 detects Host1 and advertises H1 with seq #0
3. Other VTEPs learn about the host route of Host 1
EVPN Control
VXLAN Plane ---
BGP Control VM Mobility
Plane
NLRI: Spine
• Host H-MAC-1, H-IP-1
• NVE VTEP-3
• VNI 5000
Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf
• Cost
• Sequence number: 1 Host 1
H-MAC-1
H-IP-1 MAC IP VNI Next-Hop Encap Seq#
VLAN 10
VXLAN 5000 H-MAC- H-IP-1 5000 VTEP-3 VXLAN 1
1
1. Host 1 moves to VTEP-3 from VTEP-1
2. VTEP-3 detects Host 1, sends MP-BGP update for Host 1 with its own VTEP address and a new seq #1
3. Other VTEPs learn about the new route of Host 1 from VTEP 3 with a higher sequence number and prefer
that update
EVPN Control Plane --- ARP Suppression
Minimize flood-&-learn behavior for host learning
MAC IP VNI Next- Encap Seq Spine

Hop
H-MAC- H-IP-2 5000 VTEP-3 VXLAN 0
2
2 VTEP VTEP VTEP VTEP

VTEP-1 receives and intercepts the ARP 1 2 3 4 Leaf
Request. Checks in its own host table.
• If it has an match for H-IP-2, it’ll send ARP Host 1 Host 2
response on behave of Host-2 H-MAC1 H-MAC-2
H-IP 1 H-IP-2
• If it doesn’t have a match for H-IP-2, it’ll VLAN 10 VLAN 10
forward the ARP request to remote VTEPs VXLAN 5000 VXLAN 5000
via multicast encap or head-end replication
1
Host-1 sends ARP
Request for H-IP-2
Functions of VXLAN/EVPN
Host/Network Advertise host/network reachability information

Reachability through control protocol (MP-BGP)
Advertisement
VTEP Security & Authenticate VTEPs through BGP peer
Authentication authentication
Distributed
Seamless and Optimal vm-mobility
Anycast Gateway
Early ARP termination
ARP Suppression Localize ARP learning process
Minimize network flooding
Design Options
and Use case
VXLAN Fabric Design with MP-iBGP EVPN
RR RR Spine
MP-iBGP Sessions
VXLAN Overlay
MP-iBGP EVPN
VTEP
VTEP VTEP
VTEP VTEP
VTEP VTEP
VTEP VTEP VTEP Leaf
• VTEP Functions are on leaf layer

• Spine nodes are iBGP route reflector
• Spine nodes don’t need to be VTEP
VXLAN Fabric Design with MP-eBGP EVPN
AS 65000
Spine
MP-eBGP Sessions
VTEP VTEP VTEP VTEP VTEP VTEP

Leaf
AS 65001 AS 65002 AS 65003 AS 65004 AS 65005 AS 65006
• VTEP Functions are on leaf layer

• Spine nodes are MP-eBGP Peers to VTEP leafs
• Spine nodes don’t need to be VTEP
• VTEP leafs can be in the same or different BGP AS’s
VXLAN Fabric - External Routing
VXLAN Overlay
Spine EVPN VRF/VRFs Space
VXLAN Overlay
EVPN MP-BGP Border Leaf
VTEP
VTEP VTEP VTEP VTEP VTEP
Leaf
Routing
Protocol
of
Choice Global Default VRF
Or User Space VRFs
IP Routing
VXLAN Fabric – Service Insertion
Firewall as a default gateway : Centralized Gateway- Firewall bottleneck Transparent Firewall : Inspect and then bridge Traffic between “dirty” VLAN and “clean” VLAN
Tenant Edge Firewall: Traffic between Tenants/VRFs routed via the firewall
VXLAN Fabric – Service Insertion
• Load Balancer Integration

Border Border • Load Balancer peer with
fabric using EBGP
• Injects VIP via RHI
VXLAN EVPN
VTEP VTEP VTEP VTEP
Outside
Advertise VIP
x.x.x.x/32 Client
VXLAN Fabric – Selective Traffic Redirection
• Leverages Policy Based

Border Border
Redirect
• Inter VLAN traffic bypass

VXLAN EVPN default routing lookup
Redirect HTTP only and redirected
VTEP VTEP VTEP VTEP
• Service Redirection to
Load Balancers, Firewalls
Firewall etc.
Host A (VLAN 10) Host B (VLAN 20)
192.168.10.101 192.168.20.101
VXLAN Fabric – Centralized Route Leaking
Extranet Support
• Use Cases – Shared Services,

External External Connectivity
Network
• VRF to VRF or VRF to Default
Border Border
• Centralize Location for leaking

VXLAN EVPN routes
VRF VTEP VTEP VTEP VTEP VRF
Tenant1 Tenant2
Baremetal Baremetal Baremetal
Host A Host B Host C

192.168.10.101 192.168.20.102 192.168.30.103
VXLAN Fabric – Private VLAN over VXLAN
L2VNI 30201 L2VNI 30200 Private VLAN with VXLAN

Isolate Spine Spine Community
• Extending Private VLAN over VXLAN
• Sub-VLAN Segmentation
• Availability of 2nd VLAN Modes
• Community VLAN across VXLAN
VTEP VTEP VTEP VTEP
• Promiscuous VLAN across VXLAN
• Isolate VLAN localized but extended
across VXLAN
Server Server Server Server
VLAN 201 VLAN 200

Isolate Community
VXLAN Fabric – VXLAN Pseudo wire(Xconnect)
VXLAN Spine Spine

Pseudo-Wire VXLAN Pseudo-Wire
• Cross-Connect (X-Connect) concept
• Point-2-Point
• Enables Protocol Tunneling for
VTEP VTEP VTEP VTEP
Slow Protocols
• STP, CDP, LLDP, PAGP, LACP, BFD
Switch 1 Switch 2
Peerlink-Less VPC
Enhanced dual-homing solution

without wasting physical ports
Preserve traditional vPC characteristics
VXLAN Fabric – Tenant Routed Multicast
Spine Spine
VXLAN EVPN
VRF VTEP VTEP VTEP VTEP

Tenant1 DR DR DR DR
Baremetal Baremetal Baremetal Baremetal Baremetal Baremetal Baremetal
SRC-10 RCVR-10 RCVR-20 SRC-99 RCVR-30 RCVR-11 RCVR-40

239.10.10.10 10.10.10.10 10.20.20.20 239.10.10.99 10.30.30.30 10.10.10.11 10.40.40.40
10.10.10.100 10.30.30.199
VXLAN EVPN Multi-Site
Site 1 Site 2
Border Border
Gateways Gateways
Site 1 VXLAN Tunnel Overlay Multi-Site Site 2 VXLAN Tunnel
Scale through Hierarchical Forwarding
Convergence
Fault Separate Admin
independent of Single Box
Containment Domains
Network Size
Summary
Summary
• VXLAN enables scalable Data Center fabrics

• BGP EVPN with VXLAN provides a robust control plane enabling multi-tenancy, VM
mobility , optimizing traffic forwarding
• Seamless integration with service nodes such as Firewalls and Load balancers and
ability to provide shared services
• Fabric can cater to multicast traffic in the overlay
• VXLAN as a DCI with Multi-Site
Keynote Tuesday
09:30
Friday
BRKDCN-2249 Wednesday
BRKDCN-2304 Thursday BRKDCN-3378 09:00
08:30 08:30 Building
Tuesday VXLAN VPC VXLAN
BRKDCN-2458 11:00 L4/L7 Services
VXLAN Datacenters
NXOS Operations
& practices
Wednesday
BRKDCN-2035 11:00 Friday
VXLAN Thursday BRKDCN-3346 11:30
Multi-Site BRKDCN-2025 11:15 End to End QoS
Tuesday Automation of
BRKDCN-1645 14:30 NXOS Fabrics
Introduction
VXLAN BRKDCN-2218 Wednesday
14:45 BRKDCN-2712 14:45
Mid-Size DC Insights
BRKDCN-1687 Tuesday Design
15:45 w/Analytics
Introduction
DCNM
BRKDCN-3001 Wednesday
16:45 Keynote Thursday
Micro-Segmentation 17:00
Tuesday in the DC
BRKDCN-2939 17:00
Cisco Live
Easy NXOS Fabric Celebration
(VXLAN) 18:30
NXOS Fabric Technology

#CLEMEA
Building your VTEP
FYI
(VXLAN Tunnel End-Point)
# Features & Globals Enables VTEP (only required on Leaf or Border)

feature bgp
feature nv overlay
nv overlay evpn
Enables EVPN Control-Plane in BGP
# Spine (S1)
# Leaf (V1) Configure the VTEP interface

interface nve1
RR RR RR RR
source-interface loopback0
host-reachability protocol bgp Use a Loopback for Source Interface
iBGP
V1 V2
Enable BGP for Host reachability
V3
*Simplified BGP configuration; would have 4 BGP peers (RR)
IGP not shown
Building your EVPN MP-BGP Control-Plane
FYI
# Features & Globals

feature bgp
feature nv overlay
nv overlay evpn
Enables EVPN Control-Plane in BGP
# Spine(S1)
router bgp 65500
router-id 10.10.10.1
address-family l2vpn evpn RR RR RR RR
neighbor 10.10.10.0/24 remote-as 65500 Activate L2VPN EVPN under each BGP neighbor
address-family l2vpn evpn iBGP
send-community both V2
V1
route-reflector-client
# Leaf (V1)
router bgp 65500
router-id 10.10.10.10 Send Extended BGP Community
address-family ipv4 unicast to distribute EVPN route attributes V3
neighbor 10.10.10.1 remote-as 65500
send-community both
*
Extend your VLAN to VXLAN
FYI
• VLAN to VNI configuration on a per-Switch

# Features
based feature vn-segment-vlan-based
• VLAN becomes “Switch Local Identifier” # VLAN to VNI mapping (MT-Lite)

Vlan 10
• VNI becomes “Network Global Identifier” vn-segment 5010
VLAN to Layer-2 VNI mapping
• 4k VLAN limitation per-Switch does still # Activate Layer-2 VNI for EVPN
apply evpn
vni 5010 l2 Enables EVPN Control-
• 4k Network limitation has been removed rd auto Plane for Layer-2
route-target import auto Services
• VLAN can be port-significant. The same vlan route-target export auto
on different ports can be mapping to Alternative is to use
# Activate Layer-2 VNI on VTEP “ingress-replication
different VNIs. interface nve1
VLAN
protocol bgp”
host-reachability protocol bgp
ethernet VLAN VNI vxlan member vni 5010
mcast-group 239.239.239.100
Multi-Tenancy Lite (MT-Lite) suppress-arp Enables Layer-2 VNI
on VTEP and suppress
ARP
Distributed Anycast Gateway for Extended VLANs
FYI
• All VTEPs in a VXLAN are the distributed anycast gateway for its IP subnet.
• All VTEPs in a VXLAN need to be configured with an identical anycast gateway virtual MAC
address
• All VTEPs in a VXLAN need to be configured with an to
# VLAN identical anycast gateway virtual IP address
VNI mapping
vlan 200
vn-segment 5200
# Anycast Gateway MAC, identically configured on all

VTEPs
fabric forwarding anycast-gateway-mac 0002.0002.0002
# Distributed IP Anycast Gateway (SVI)

One gateway virtual MAC per VTEP # Gateway IP address needs to be identically
configured on all VTEPs
interface vlan 200
no shutdown
vrf member VRF-A
ip address 20.0.0.1/24
One gateway virtual IP per VLAN/VXLAN fabric forwarding mode anycast-gateway
Routing in VXLAN – Define the Resources
FYI
Configuration Example for VRF-A

# Define VLAN for VRF routing instance
Layer-3 VNI
Vlan 999
(VNI 9999 / VLAN 999) VLAN to Layer-3 VNI mapping
vn-segment 9999
# Define SVI for VRF routing instance

Layer-2 VNI interface Vlan999
Layer-2 VNI VLAN to Layer-3 VNI mapping
(Network VNI) no shutdown
(Network VNI) mtu 9216 - ip forward required for prefix-
vrf member VRF-A based routing
VTE VTE VTE VTE
P P P ip forward
P
# VRF configuration for “customer” VRF

vrf context VRF-A
vni 9999
VRF context definition
1:1 mapping between L3 VNI rd auto - VNI
and tenant VRF address-family ipv4 unicast
- Route-Distinguisher
route-target both auto
route-target both auto evpn - Route-Targets
- IPv4 and/or IPv6
Routing in VXLAN – Configure the Routing
FYI
Enables Layer-3 VNI on VTEP Configuration Example for VRF-A

andVNIassociate it to VRF # Activate Layer-3 VNI on VTEP
Layer-3
interface nve1
(VNI 9999 / VLAN 999)
member vni 5010
Layer-2 VNI mcast-group 239.239.239.100
Layer-2 VNI suppress-arp
(Network VNI)
(Network VNI) member vni 9999 associate-vrf
VTE VTE VTE VTE
P P P # Route-Map for Redistribute Subnet
P
route-map REDIST-SUBNET permit 10
match tag 12345
1:1 mapping between L3 VNI # Control-Plane configuration for VRF (Tenant)
and tenant VRF router bgp 65500
…
vrf VRF-A
VRF/Tenant definition advertise l2vpn evpn
within Overlay Control-Plane redistribute direct route-map REDIST-SUBNET
maximum-paths ibgp 2
VXLAN Hardware Gateway Redundancy
FYI
(vPC)
• Redundant connectivity for classic
Ethernet hosts
• Extend the IP Interface (Loopback)
configuration for the vPC VTEP
• Secondary IP address (anycast) is used as
the anycast VTEP address
V5
• Both vPC VTEP switches need to have the V4
identical secondary IP address configured
under the loopback interface
Host D
VNI 30000
FYI
(vPC)
vPC VTEP Configuration Example
# VLAN to VNI mapping (MT-Lite)
vlan 55 interface loopback0
vn-segment 30000 ip address 10.10.10.5/32
ip address 10.10.10.99/32 secondary
# VTEP IP Interface; Source/Destination for all
VXLAN Encapsulated Traffic.
▪ Primary IP address is used for Orphan Hosts
▪ Secondary IP is for vPC Hosts (same IP on both
vPC Peers)
interface loopback0
Add Secondary IP to VTEP V5
ip address 10.10.10.5/32
ip address 10.10.10.99/32 secondary Loopback.
V4
VXLAN automatically picks up
# VTEP configuration using Loopback as source. the secondary IP address as
interface nve1 the VTEP address
interface loopback0
member vni 5010 ip address 10.10.10.4/32
mcast-group 239.239.239.100 ip address 10.10.10.99/32 secondary
suppress-arp Host D
member vni 9999 associate-vrf VNI 30000
FYI
(vPC)
vPC VTEP Configuration Example
# VPC Domain Configuration
vpc domain 99 interface loopback0
peer-gateway needs to be ip address 10.10.10.5/32
peer-switch
enabled so that vPC VTEP ip address 10.10.10.99/32 secondary
peer-keepalive destination V4-mgmt source v5-mgmt
peer-gateway
switches can forward traffic
ip arp synchronize for each other’s router MAC
address
# VPC Peer-Link
interface port-channelXX V5
switchport mode trunk
vpc peer-link
V4
# VPC Domain Routing Adjacency Routed Interface (SVI) for routing
interface Vlan3999
no shutdown
adjacency across VPC Peer-Link
ip address 10.254.254.1/30
interface loopback0
ip router ospf 1 area 0.0.0.0 ip address 10.10.10.4/32
ip ospf network point-to-point ip address 10.10.10.99/32 secondary
ip pim sparse-mode Host D
VNI 30000
eBGP EVPN Configuration (1)
FYI
Next-hop Unchange
• BGP next-hop is used as the tunnel tail
end address. It shall be the advertising
eBGP configuration on a spine switch
VTEP’s address.
• Ensure the next-hop in the BGP route route-map permit-all permit 10
isn’t changed during the route route-map nh-unchange permit 10

set ip next-hop unchanged
distribution router bgp 65000
• eBGP changes next-hop by default. router-id 10.1.1.1
Need to change the policy to next-hop address-family ipv4 unicast

unchanged nexthop route-map nh-unchange
retain route-target all
neighbor 192.167.11.2 remote-as 65001
Set next-hop policy not to change address-family ipv4 unicast
the next-hop attribute address-family l2vpn evpn
send-community extended
route-map permit-all out
eBGP EVPN Configuration(2)
FYI
Manually configure import/export route-target
• With eBPG, VTEPs will have different

route-targets if using auto RT vrf context evpn-tenant-1
generation vni 9999
rd auto
• Need to manually configure RTs on address-family ipv4 unicast
eBGP peers so that they have the same route-target import 100:9999
RTs
route-target import 100:9999 evpn
route-target export 100:9999
route-target export 100:9999 evpn
evpn
Manually configure route-target for vni 5010 l2
VRF rd auto
route-target import 100:5010
route-target export 100:5010
Manually configure route-target for

L2 VNI under EVPN
Complete your
online session
survey • Please complete your session survey
after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live t-shirt.
• All surveys can be taken in the Cisco Events
Mobile App or by logging in to the Content
Catalog on ciscolive.com/emea.
Cisco Live sessions will be available for viewing on

demand after the event at ciscolive.com.
Continue your education
Demos in the
Walk-in labs
Cisco campus
Meet the engineer

Related sessions
1:1 meetings
Thank you

BRKDCN 1645

Uploaded by

Copyright:

Available Formats

You might also like

BRKDCN 1645

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BRKDCN 1645

Uploaded by

Copyright:

Available Formats

Introduction to VXLAN

The future path of your

Rahul Parameswaran, Technical Marketing Engineer

• Use Cisco WebEx Teams for Questions

• Watch out for the hidden slides ☺

• A short overview on Data Center Evolution

▪ Data Center evolution

Layer-3 HSRP HSRP

Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2 Layer-2

Spine Spine Spine Spine

Customer Needs VXLAN Delivered

VM Mobility Seamless VM Mobility

Secure Multi-tenancy Traffic & Address Isolation

Identifier = VN Identifier (VNID) Overlay Control Plane

Edge Devices (NVE)

• VXLAN uses MAC in UDP encapsulation (UDP destination port 4789)

MAC MAC 802.1Q IP DA IP SA UDP ID MAC MAC l Inner Ethernet

VXLAN Encapsulation Original Ethernet Frame

AnyCast RP ip pim rp-address 10.237.1.1 group-list 224.0.0.0/4

ip pim rp-address 10.237.1.1 group-list

Flood-and-Learn VXLAN: VXLAN EVPN:

• Limited scale • Increased scale and stability

BUM Traffic transport mechanisms

make VPN IP routes unique: RD + VPN IP prefix

Route Target (RT): 8-byte field, VRF parameter,

for VPNv4 routes RDroute-target

• Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)

Provider Backbone Bridges

Switch level multi-tenancy

Tenants or VRF for L3 logical separation

VTEP VTEP VTEP VTEP

RP Rendezvous Point (Underlay)

! spine bgp config

router bgp 65001

• EVPN Control Plane -- Host and Subnet Route Distribution

▪ Use MP-BGP with EVPN Address Family on leaf nodes to distribute

interface Vlan10 Host 1 Host 2 Host 3

# VLAN to VNI mapping

# Anycast Gateway MAC, identically configured on all VTEPs

# Distributed IP Anycast Gateway (SVI)

Host 1 Host 2 Host 3 Host 4

Dst MAC: MAC-A 5 Outer S-IP: IP-2 ARP Request for IP B

BGP EVPN Type-2 MAC +Host IP update

3. Other VTEPs learn about the host route of Host 1

MAC IP VNI Next- Encap Seq Spine

2 VTEP VTEP VTEP VTEP

Host/Network Advertise host/network reachability information

• VTEP Functions are on leaf layer

VTEP VTEP VTEP VTEP VTEP VTEP

AS 65001 AS 65002 AS 65003 AS 65004 AS 65005 AS 65006

• VTEP Functions are on leaf layer

• Load Balancer Integration

VTEP VTEP VTEP VTEP

• Leverages Policy Based

• Inter VLAN traffic bypass

• Use Cases – Shared Services,

• Centralize Location for leaking

Baremetal Baremetal Baremetal