Dcevpn 1619196632181

Click to edit Master title style
Goals & Basics
Russ
A E
B F
C D
A E
B F
C D
A E
B F
C D
A E
B F
C D
Standards
• Requirements for Ethernet VPN (EVPN)
• RFC7209
• A Network Virtualization Overlay Solution Using
Ethernet VPN (EVPN)
• RFC8365
• Usage and Applicability of BGP MPLS-Based Ethernet
VPN
• RFC8388
Drafts
• EVPN multi-homing port-active load-balancing
• draft-ietf-bess-evpn-mh-pa
• EVPN Operations, Administration and Maintenance Requirements
and Framework
• draft-ietf-bess-evpn-oam-req-frmwk
• Preference-based EVPN DF Election
• draft-ietf-bess-evpn-pref-df
• IP Prefix Advertisement in EVPN
• draft-ietf-bess-evpn-prefix-advertisement
• Weighted Multi-Path Procedures for EVPN All-Active Multi-Homing
• draft-ietf-bess-evpn-unequal-lb
eVPN and the Underlay
Russ
A B C D E
A B C D E
VxLAN Click to edit Master title style
https://tools.ietf.org/html/rfc7348
A B C D E F G H
1
A B C D
2
2
3
1
0
5
Route Types
Jeff
Basic Route Types
• RT-2 advertises MAC and MAC/IP(host)
• RT-3 advertises EVI membership (IMET)
• RT-5 - advertises IP prefix
EVPN Route Types
MP_REACH_NLRI
Flags
Type (MP_REACH_NLRI) eVPN Route Types
Length 1: Ethernet A-D
AFI=25 2: MAC Advertisement
SAFI=70 3: Inclusive Multicast
Next-hop Address (VTEP IP Address) 4: Ethernet Segment
5: IP Prefix
Route Type
Route Length
Route Type Specific Fields

Route Type 2: MAC/IP Advertisement
Path Attribute MP_REACH_NLRI Type 2 NLRI
Next-hop IP=VTEP IP Route Distinguisher (RD)
AFI=25 (L3VPN), SAFI=70 (eVPN) Ethernet Segment Identifier
Ethernet Tag ID
Route Type (IP Prefix, Type2) MAC Address Length (48)
Length MAC Address
IP Address Length (optional)

1: Ethernet A-D IP address (optional)
2: MAC-Advertisement
3: Inclusive Multicast MPLS Label (VNI)
4: Ethernet Segment
5: IP Prefix (optional)
Extended Community Route Target
Router MAC’s Extended Community
(optional)
Extended Community MAC Mobility
Tunnel Encapsulation Extended
Community
Junos EOS
Junos EOS
Route Type 3: Inclusive Multicast
Path Attribute MP_REACH_NLRI Type 3 NLRI
Next-hop IP=VTEP IP NLRI
AFI=25 (L3VPN), SAFI=70 (eVPN) Route Distinguisher (RD) RD of the advertising NVE
Ethernet TAG VLAN aware=0; VNI for bundle

Route Type (IP Prefix, Type3) IP Address Length IPv4/IPv6
Length Router IP Address NVE IP Address
1: Ethernet A-D
2: MAC-Advertisement Extended Community RT RT of the MAC-VRF or EVI
3: Inclusive Multicast
4: Ethernet Segment Tunnel Encap Community VXLAN
5: IP Prefix (optional)
PMSI Tunnel Attribute Multicast or Ingress
Junos
EOS
Route Type 5: IP Prefix Route
IPv4 Basic RT-5 Interface-less RT-5
Path Attribute MP_REACH_NLRI Path Attribute MP_REACH_NLRI
Next-hop IP is VTEP-IP Next-hop IP is VTEP-1
Type 5 Route Type 5 Route

Route Distinguisher (RD) Route Distinguisher (RD)
Ethernet Segment ID Ethernet Segment ID (0)
Ethernet TAG (0 for VLAN-based
Ethernet TAG
service)
IP Address Length IP Address Length (Prefix Mask)
IPv6 IP Address IP Address (Subnet-A)
Gateway IP Address Gateway IP Address (0)
VNI Label VNI Label (IP-VRF) (2000)
Route Target Extended Community

Router Target Extended Community
(2000:2000)
Router Target Extended Community
Router MAC Extended Community
(MAC-A)
Tunnel Encap Extended Community
Tunnel Encap Extended Community
(VXLAN)
Click to edit Master
Junos title style
EOS
Multihoming Route Types
• RT-1 per ES – used for Mass Withdrawal
• RT-1 per EVI – used for aliasing, load-sharing
• RT-4 – DF election, AD of multihomed ES's
Route Type 1
Ethernet Auto-discovery Route
Route Type 4
Ethernet Segment Route
ESI types
T (ESI Type) is a 1-octet field (most significant octet) that
specifies the format of the remaining 9 octets (ESI Value)
Multicast Route Types
• RT-6 – Selective Multicast Route (SMET)
• RT-7 – IGMP/MLD Join Synch Route
• RT-8 – IGMP/MLD Leave Synch Route
EVPN service interfaces
Broadcast and Unknowns

BUM Reduction
• Large flooding domains are problematic
• Timing issues with L2 stretched across multiple sites
• 600k hosts/10min ARP cache timeout == 1k pps of ARP
• Normally processed by the router CPU
• Other broadcast/multicast traffic
• Restrict traffic flow to correct subset of connected hosts
• Optimizing replication
1 2 3 4
A E
G
H
B F
A ARP to E
C
1 2 3 4
A E
G
H
B F
E response to ARP
C D
1 2 3 4
A E
G
H
B F
C D
C ARP to A
1 2 3 4
A E
G
H
B F
A ARP cache timeout

C D
1 2 3 4
A E
G
H
B F
standard multicast replication

C D
1 2 3 4
A E
G
H
B F
selective multicast ethernet tag

C D
Other Multicast Optimizations
• Many other multicast optimizations being discussed and
implemented
• P2MP multicast distribution
• Assisted Replication
• Optimized inter-subnet multicast
• Current Drafts
• draft-ietf-bess-evpn-igmp-mld-proxy
• draft-ietf-bess-evpn-optimized-ir
• draft-ietf-bess-evpn-irb-mcast
• draft-skr-bess-evpn-pim-proxy
• draft-surajk-evpn-access-security
Segmentation
L2 segmentation
• Control plane
• Similar to L3VPN
• Provides uniqueness/multitenancy
• RD is prepended to MAC address, RD:MAC
• Data plane(L2VNI):
• Each VXLAN segment is identified through a 24-bit
segment ID, termed the "VXLAN Network Identifier
(VNI)"
• Allows up to 16M VXLAN segments to coexist within the
same administrative domain
L3 segmentation
• Control plane
• Similar to L3VPN
• Provides uniqueness/multitenancy
• RD is prepended to IP address, RD:IP
• Data plane(L3VNI):
• Each VXLAN segment is identified through a 24-bit
segment ID, termed the "VXLAN Network Identifier
(VNI)“
• Allows up to 16M VXLAN segments to coexist within the
same administrative domain
L3 – Asymmetric IRB
L3 - Asymmetric IRB
Asymmetric IRB performs bridging and routing on the ingress VTEP, but only bridging on the egress
VTEP.
Asymmetric IRB may sometimes be described as bridge-route-bridge. This refers to the lookups
performed when moving traffic between two layer 2 segments. The ingress VTEP performs a
bridging and routing operation, while the egress VTEP only performs a bridging operation.
1 2 3 4
A E
G
L3 - Asymmetric IRB
bridge-route-bridge
H
B F

C D
L3 – Symmetric IRB
L3 - Symmetric IRB
In Symmetric IRB, there is a dedicated Layer 3 VNI that is used for all layer 3 routing between any two layer
2 VNIs for the same tenant. This results in more configuration for the devices, and it also requires an
additional hardware lookup when compared to Asymmetric IRB, but it is more scalable. Needed for
communicaiton with the external world!!!
Symmetric IRB may sometimes be described as bridge-route-route-bridge. This refers to the

ingress VTEP performing a bridging and routing operation and then the egress VTEP routing and
bridging
1 2 3 4
A E
G
L3 - Symmetric IRB
bridge-route-route-bridge
H
B F

C D
Mobility
1 2 3 4
A E
G
H
B F
E (old) D
1 2 3 4
A E
G
H
B F
E (old) D
1 2 3 4
A E
G
H
B F
C (new) D
MAC Mobility Dampening
• When advertising a mobility event
• Set a mobility timer (M)
• If a lot of mobility events detected
• Log
• Flush route
• Stop processing updates
1 2 3 4
A E
G
H
B F
C (pinned) D
1 2 3 4
A E
G
H
B F
C (pinned) D
ESI Multi-homing
1 2 3 4
A E
G
5 6 7 8
H
B F
ES
C
1 2 3 4
A E
G
5 6 7 8
H
B F
LACP/Static ES LACP/Static
C
1 2 3 4
A E
G
Route Type 4
Ethernet Segment Route – DF election
5 6 7 8
H
B F
ES
C
1 2 3 4
A E
G
Route Type 1 per ES

Ethernet Auto-discovery Route – mass withdraw
5 6 7 8
H
B F
ES
C
1 2 3 4
A E
G
Route Type 1 per EVI

Ethernet Auto-discovery Route – aliasing
5 6 7 8
H
B F
ES
C
1 2 3 4
A E
G
Route Type 2
MAC or MAC/IP with ESI set
5 6 7 8
H
B F
ES
C
1 2 3 4
E
RT1 per ES(6-7):
ESI=00::01
A ETH-TAG=MAX-ET
All-active
G RT1 per EVI(6-7):
ESI=00::01
ETH-TAG!=MAX-ET
RT2(6 or 7):
ESI=00::01
MAC=C
RT4
5 6 7 8
H
B F
ES ESI=00::01
compressed 00:00:00:00:00:00:00:00:00:01
C
Basic Deployment
L2
1 2 3 4
L2 only VRRP
Intra-subnet Proxy ARP
Df-GW=VRRP-IP
192.168.1.x/24
5 6 7 8
L2
F
C D
1 2 3 4
SVI/IRB anycast
L2/L3
Inter-subnet
192.168.1.x/24
SVI/IRB anycast
192.168.2.z/24
5 6 7 8
C D
1 2 3 4
L3
Inter-subnet
192.168.1.x/32
192.168.1.x/32
5 6 7 8
192.168.2.x/24
F
C D
Data Center Interconnect

Over the Top (similar to RFC4364 Inter-AS option C)
requires coordination of VNI's and RD/RT's
EVPN AFI/SAFI
Unicast v4/v6 AFI/SAFI
WAN
IP/MPLS
DCI using GW - Independent Control Planes
RFC 8365, section-10.1, VNI/RT translation
VXLAN EVPN VXLAN EVPN
EVPN/L3VPN/L2VPN
WAN
IP/MPLS
DCI using ASBR(similar to RFC4364 Inter-AS option B)
RFC 8365, section-10.2, VNI/RT translation
VXLAN EVPN VXLAN EVPN
MPLS EVPN
WAN
IP/MPLS
Routing on the Host

1 2 3 4
A
G
H
B
C D
1 2 3 4
A
G
H
B
C D
1 2 3 4
A
G
H
B
C D
Troubleshooting
Troubleshooting - underlay
• For overlay to work – underlay must be fully functional
• Always start troubleshooting with underlay!
• Underlay routing must work properly
• VTEP IP's must be distributed
• BGP infra IP's must be distributed
• Number of implementations requires VTEP source != BGP
source – pay attention, your overlay might not work
• MTU – VXLAN encapsulation adds 50(54 with
802.1q)bytes. Make sure core facing interfaced are
configured to support additional overhead (=Jumbo)
Troubleshooting – EVPN
• Start with basic BGP:
• make sure EVPN capability has been exchanged between
peers and the session is "Established"
• On spines/super-spine
• Make sure next-hop unchanged has been set
• Make sure routes that have no local import are still
advertised (NX-OS retain route-target)
• Make sure import/export policies have been configured
(Junos)
Troubleshooting – EVPN, cont'd
• EVPN EVI (MAC VRF)
• Make sure RT import/export policies within respective
EVI's match
• Make sure to understand the differences between
different EVPN SI's and how they manifest in routes
• Make sure to understand how host routes(ARP/ND) are
represented (type 2 L3VNI vs type 5), differs per vendor
• If using MLAG/vPC – make sure to follow vendor's
specifics, every vendor has their own
• Every vendor has their own logic wrt RD generation
• EVPN EVI (MAC VRF)
• After an EVI has been configured – type 3 route for that
EVI must appear on every switch participating
• You will see 2 RT-2 per ARP/ND, MAC only + MAC/IP
• Mind vendor implementations:
• NX-OS/EOS/Cumulus - L2VNI/RT+L3VNI+RT
• Junos/Nokia - L2VNI/RT only
• EVPN VRF(IP VRF)
• Every unicast route imported (redistributed) into BGP
and exported becomes RT-5 (usually SVI's)
• It is possible to build RT-5 only design (ala L3VPN)
• In DCI cases – don't stretch L2 (unless absolutely
necessary), leak RT-5 only
• VRF-leaking – remember, leaked routes are not re-
advertised back into EVPN fabric (local only)
• EVPN ESI multihoming
• After ES has been configured and up:
• 1 RT-1 per ES per switch
• 1 RT-1 per EVI per ES per switch
• Mind vendors/EVNP SI
• Junos VLAN Aware SI – RT-1 ES = RT-1 EVI
• 1 RT-4 per ES per switch, imported only by ES owners
• Import RT (ES-Import) is derived from ESI and is auto-
generated
• Mind EOS – RT is manually configured
Telemetry Considerations
Self-healing
TE
Prov.
TSDB
Cent. CP
Orch.
DB
Analytics
DB
Dist. CP
DB
Apps
Set Goals
• Notify on link down, up, etc.
• Notify on changes in reachable destinations
• Notify on node failure
• Reconstruct the state of the network at the time of a
failure
• Decrease MTTR
• Understand “normal”
• Understand where to look to find problems quickly
Interface Errors
Predictive BGP Peer uptime
Decrease MTTR VRF Route Count
Route count
VRF Errors Membership

NOTIFICATION
Current
BGP Peer State Peer timeout
MTU Mismatch
Interface Errors
Down
Click to edit Master title style Change in delay from reroute
Jitter from reroute

Application Failure
Packet loss from reroute
Loss of VRF route count
BGP peer down
Interface Down
Neighbor Count
VRF IP Route Count
Underlay: topology is correct
Overlay: virtual topology is correct
VRF MAC Destination Count

ECMP Path Count
Number of eVPN Instances Overlay: virtual topology is correct
(EVIs)
ECMP Balance Correct traffic distribution in underlay and overlay
NLRI Received Count Normal versus change
NLRI Sent Count
BGP Peer Uptime
BUM Traffic Rate
eVPN DF Election Correlation with other problems
MAC Address Move

Dcevpn 1619196632181

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dcevpn 1619196632181

Uploaded by

Copyright:

Available Formats

Click to edit Master title style

Goals & Basics

eVPN and the Underlay

Type (MP_REACH_NLRI) eVPN Route Types

Length 1: Ethernet A-D

AFI=25 2: MAC Advertisement

SAFI=70 3: Inclusive Multicast

Next-hop Address (VTEP IP Address) 4: Ethernet Segment

Route Type Specific Fields

IP Address Length (optional)

Ethernet TAG VLAN aware=0; VNI for bundle

Type 5 Route Type 5 Route

Route Target Extended Community

Broadcast and Unknowns

A ARP cache timeout

standard multicast replication

selective multicast ethernet tag

selective multicast ethernet tag

Symmetric IRB may sometimes be described as bridge-route-route-bridge. This refers to the

selective multicast ethernet tag

Route Type 1 per ES

Route Type 1 per EVI

Data Center Interconnect

Unicast v4/v6 AFI/SAFI

VXLAN EVPN VXLAN EVPN

VXLAN EVPN VXLAN EVPN

Routing on the Host

Predictive BGP Peer uptime

Decrease MTTR VRF Route Count

VRF Errors Membership

BGP Peer State Peer timeout

Jitter from reroute

Packet loss from reroute

Loss of VRF route count

BGP peer down

VRF MAC Destination Count

You might also like