Cisco Vxlan Config v1

Lab Guide
Cisco dCloud
Cisco VXLAN Configuration v1
Last Updated: 23-February-2021
Created by Solutions Readiness Engineers
About This Demonstration

This guide for the preconfigured Cisco VXLAN Configuration v1 lab guide includes:
About This Demonstration
Requirements
About This Solution
Topology
Get Started
Scenario 1. Build the VXLAN Underlay
Scenario 2. Configuring the BGP Underlay
Scenario 3. Configuring Overlay
Scenario 4. Configure Host Connections
Scenario 5: External Routing
Appendix A. Device Toubleshooting
What’s Next?
© 2021 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 1 of 38
Lab Guide
Cisco dCloud
Limitations
Certain features of the Cisco VXLAN solution are outside the scope of this demonstration, because the
demonstration uses virtual devices rather than a physical fabric:
• Due to the way the Nexus 9000v operates, it does not start with a boot statement and will get stuck in
loader on boot. To prevent this, make sure the show boot command contains a valid image to boot from.
• Some commands are not available on the virtual Nexus 9k that might be required on a CloudScale Nexus
9K. Please consult the documentation.
• Since the hardware is virtual, some things may appear odd when it comes to the interfaces. For example, if
two interfaces are directly connected, shutting one side down should show "down" on the other side. This
does not occur in virtual hardware.
Customization Options
We recommend that you test different scenarios after building the VXLAN Fabric.
• Tenant-2 is built but is not really used throughout the demo. We recommend that you move some of the
servers to Tenant-2 to show how multi-tenancy works to isolate traffic.
• The vPC configuration is very generic. It is outside the scope of this lab to set up the vPC configuration with
"advertise-pip". We recommend that you try and play with it. It does work, and it is helpful to know the
differences.
Lab Guide
Cisco dCloud
Requirements
The table below outlines the requirements for this preconfigured demonstration.
Required Optional
Laptop Cisco AnyConnect®
Router, registered and configured for Cisco

dCloud
About This Solution

VXLAN is essentially an overlay that transports Layer 2 frames across a Layer 3 IP Network. It provides the
ability to scale a Layer 2 broadcast domain at a larger scale while utilizing the full bandwidth of the network
without blocking ports due to spanning-tree. It provides the following advantages
• Uses a Layer 3 network to tranfer Layer 2 frames
• Allows for ECMP through the Spine Switches
• Limits spanning-tree to the host facing Leaf switch ports

• Uses BGP to transfer the Layer 2 reachability information
• Provides Multi-tenancy to have different customer networks reside in the same fabric
Lab Guide
Cisco dCloud
Topology
This content includes preconfigured users and components to illustrate the scripted scenarios and features of
the solution.
dCloud Topology
Physical Topology
Lab Guide
Cisco dCloud
Equipment Details
Name Description Host Name (FQDN) IP Address Username Password
CML Cisco Modeling Labs cml.dcloud.cisco.com 198.18.133.3 guest C1sco12345
Spine-1 Nexus 9K admin C1sco12345
Spine-2 Nexus 9K admin C1sco12345
Leaf-1 Nexus 9K admin C1sco12345
WAN Router IOSv admin C1sco12345
Server-1 Ubuntu Image cisco cisco
Server-2 TinyCore Linux cisco cisco
Switch Information
Loopback 15
Name Loopback 0 IP Loopback 1 IP Loopback 1 Secondary
IP
Spine-1 10.0.0.1 10.0.1.1 10.255.255.255
Spine-2 10.0.0.2 10.0.1.2 10.255.255.255
Leaf-1 10.0.0.11 10.0.1.11 10.0.1.100
Leaf-2 10.0.0.12 10.0.1.12 10.0.1.100
Leaf-3 10.0.0.13 10.0.1.13
Leaf-4 10.0.0.14 10.0.1.14
Lab Guide
Cisco dCloud
Tenant Information
Name VLAN ID VLAN Name VNI Multicast Group SVI IP
Tenant-1 101 Tenant-1_Network-1 10101 239.0.0.101 192.168.101.1/24
Tenant-1 1001 Tenant-1_L3VNI 101001 N/A N/A
Tenant-2 1002 Tenant-2_L3VNI 101002 N/A N/A
Server Information
Name VLAN IP Address Gateway
Server-1 101 192.168.101.10/24 192.168.101.1
Server-2 101 192.168.101.20/24 192.168.101.1
Server-3 102 192.168.102.30/24 192.168.102.1
Server-4 N/A 172.16.3.40/24 172.16.3.1
Component Details
• CML - 2.1.1-b19
• Nexus 9K - 9.3(6)
• IOSv - 15.9(3)M2
• Ubuntu - 20.04.1
• TinyCore Linux - 5.4.3-tinycore
Lab Guide
Cisco dCloud
Get Started
Follow these steps to schedule a session of the content and configure your presentation environment.
1. Initiate your dCloud session. [Show Me How]
NOTE: It may take up to 10 minutes for your session to become active.
2. For best performance, connect to the workstation with Cisco AnyConnect VPN [Show Me How] and the
local RDP client on your laptop [Show Me How]
• Workstation 1: 198.18.133.252, Username: administrator, Password: C1sco12345.
IMPORTANT! After you access the remote desktop, wait 15 minutes for the devices to fully initialize. If you
do not wait accordingly, the devices may not be accessible.
This demonstration/lab is designed to be completed in one sitting without interruption, otherwise you may see
some errors and may have to log back into the application and/or devices.
NOTE: The Nexus 9000v I/O is demanding of dCloud platform resources. As a result, device crashes may
occur. To recover failed devices, refer to the Device Troubleshooting Appendix in this document.
Lab Guide
Cisco dCloud
Accessing Devices
1. On the remote windows desktop, double-click the devices folder.

2. Double-click a Leaf, Spine, or server icon to launch the device and access its command line interface
(CLI). Note that it can take several moments for a terminal session to fully load. If you receive a security
warning when launching a device, click Yes to continue.
Lab Guide
Cisco dCloud
Scenario 1. Build the VXLAN Underlay

Value Proposition: In this scenario, we will build the routing tables using OSPF. We will utilize IP Unnumbered
on the interfaces in order to minimize the amount of IP Addresses we must use. The feature PIM also has to be
enabled. In order to successfully pass BUM traffic, either multicast or ingress replication must be used to pass
traffic such as ARP or other similar traffic between the Leaf switches. BGP, or in this case Multiprotocol BGP, is
required to pass the Layer 2 reachability information between the Leaf switches for their connected hosts. This
step will actually be covered in Scenario 2.
Build the Underlay IGP Routing Protocol
NOTE: The Nexus 9000v I/O is demanding of dCloud platform resources. As a result, device crashes may
occur. To recover failed devices, refer to the Device Troubleshooting Appendix in this document.
1. On all of the Spine and Leaf switches, enter the following commands to enable the OSPF routing protocol
and set the Router-ID to match the Loopback 0 IP address.
Spine-1:
Spine-1# configure
feature ospf
router ospf UNDERLAY
router-id 10.0.0.1
end
copy run start
Spine-2:
Spine-2# configure
feature ospf
router-id 10.0.0.2
end
copy run start
Leaf-1:
Leaf-1# configure
feature ospf
router-id 10.0.0.11
end
copy run start
Leaf-2:
Leaf-2# configure
feature ospf
router-id 10.0.0.12
end
copy run start
Lab Guide
Cisco dCloud
Leaf-3:
Leaf-3# configure
feature ospf
router-id 10.0.0.13
end
copy run start
Leaf-4:
Leaf-4# configure
feature ospf
router-id 10.0.0.14
end
copy run start
Now we will configure the interfaces for OSPF. In this setup, the goal is to enable OSPF with a point-to-point
network for faster convergence. Each of the loopback interfaces must be reachable throughout the network.
The loopbacks have already been created. In this case, the goal is to save IP Space also inside the fabric by
using “ip unnumbered”.
Loopback addresses described:
• Loopback0 – Used for the “ip unnumbered” and for the BGP Peering source/destination
• Loopback1 – Used for the VXLAN tunnel interface source and destination
• Loopback15 – Used only on spine switches for the Anycast RP address for multicast routing. Multicast
routing is used for BUM traffic discovery.
Spine-1 and Spine-2:
1. Throughout the lab, the same config can be used on multiple devices. In this setup, we recommend that you
use a text editor in order to copy and paste the configuration. In this situation, all of the Spine Switches use
the exact same config. Only Spine-1 is shown below. Make sure to put the config on both Spine-1 and
Spine-2.
Spine-1# configure
interface ethernet1/1-4
no switchport
medium p2p
ip router ospf UNDERLAY area 0.0.0.0
ip unnumbered loopback0
no shutdown
exit
interface loopback0
interface loopback1
interface loopback15
end
copy run start
Lab Guide
Cisco dCloud
Leaf-1, Leaf-2, Leaf-3, and Leaf-4:

2. Here, all of the leaf switches will utilize the same config. Make sure to put the config on all four Leaf
switches.
Leaf-1# configure
interface ethernet 1/1-2
no switchport
medium p2p
ip unnumbered loopback0
no shutdown
exit
interface loopback0
interface loopback1
end
copy run start
Verification of the Underlay IGP
It is important to verify that routing is up and working properly on the Spines/Leaf switches. The details of OSPF
are outside the scope of this lab. Please verify it’s functionality. A few of the commands to look for are shown
below.
1. Enter the following command on Spine-1 and Spine-2. In this output, we are looking to verify that all of the
Leaf switches did form an OSPF neighbor adjacency.
Spine-1:
Spine-1# show ip ospf neighbors
OSPF Process ID UNDERLAY VRF default
Total number of neighbors: 4
Neighbor ID Pri State Up Time Address Interface
10.0.0.11 1 FULL/ - 00:01:05 10.0.0.11 Eth1/1
10.0.0.12 1 FULL/ - 00:01:07 10.0.0.12 Eth1/2
10.0.0.13 1 FULL/ - 00:00:56 10.0.0.13 Eth1/3
10.0.0.14 1 FULL/ - 00:00:54 10.0.0.14 Eth1/4
Spine-1#
Spine-2:
Spine-2# show ip ospf neighbors
OSPF Process ID UNDERLAY VRF default
Total number of neighbors: 4
Neighbor ID Pri State Up Time Address Interface
10.0.0.11 1 FULL/ - 00:01:48 10.0.0.11 Eth1/1
10.0.0.12 1 FULL/ - 00:01:48 10.0.0.12 Eth1/2
10.0.0.13 1 FULL/ - 00:01:37 10.0.0.13 Eth1/3
10.0.0.14 1 FULL/ - 00:01:38 10.0.0.14 Eth1/4
Spine-2#
Lab Guide
Cisco dCloud
Spine-1:
2. Enter the following command on Spine-1. In this output, the goal is to verify that all of the loopback IP
Addresses are reachable from each device. In this example, only the view from Spine-1 is shown. It is
highly recommended to check this output on each switch (Spine 1 and 2; Leaf-1, Leaf-2, Leaf-3 and Leaf-
4.
Spine-1# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.0.0.1/32, ubest/mbest: 2/0, attached

*via 10.0.0.1, Lo0, [0/0], 00:29:47, local
*via 10.0.0.1, Lo0, [0/0], 00:29:47, direct
10.0.0.2/32, ubest/mbest: 4/0
*via 10.0.0.11, Eth1/1, [110/81], 00:03:49, ospf-UNDERLAY, intra
10.0.0.11/32, ubest/mbest: 1/0
10.0.0.12/32, ubest/mbest: 1/0
10.0.0.13/32, ubest/mbest: 1/0
10.0.0.14/32, ubest/mbest: 1/0
*via 10.0.1.1, Lo1, [0/0], 00:29:47, local
*via 10.0.1.1, Lo1, [0/0], 00:29:47, direct
10.0.1.2/32, ubest/mbest: 4/0
10.0.1.11/32, ubest/mbest: 1/0
10.0.1.12/32, ubest/mbest: 1/0
10.0.1.13/32, ubest/mbest: 1/0
10.0.1.14/32, ubest/mbest: 1/0
10.0.1.100/32, ubest/mbest: 2/0
*via 10.255.255.255, Lo15, [0/0], 00:29:47, local
*via 10.255.255.255, Lo15, [0/0], 00:29:47, direct
Spine-1#
Lab Guide
Cisco dCloud
Configure Multicast IP Routing with PIM
One of the bigger hurdles to get a VXLAN fabric working is to make sure that Broadcast, Unknown Unicast, and
Multicast traffic operates as expected. This type of traffic is also called BUM traffic.
There are actually 2 ways to solve the problem.

• Option 1 – ingress-replication using BGP. This option removes the need for Multicast on the Underlay. It
requires each Leaf Switch to be able to replicate a packet to every other leaf switch using the specific VNI
• Option 2 – Multicast. This option uses Multicast Groups to forward BUM traffic. It requires the Spine Switch
to be able to replicate a packet to all Leaf Switches listening on a multicast group
In this example, we will use Multicast. There are a handful of ways to configure multicast. For simplicity in the
configuration, we will be using Anycast RP. It involves some extra configuration on the spine switches using
Loopback15 (was previously configured) with the same IP Address on both spine switches.
For the Spine switches, each of the loopback interfaces and physical interfaces that are connected to Leaf
switches need to be configured to run “ip pim sparse-mode”. The “anycast-rp” configuration tells the switch
which IP will be the RP address and the Loopback0 for the other switches that are running the Anycast RP
Address. Finally, it needs to be told what the RP address is.
Configure Anycast RP
1. Here, both Spine switches will utilize the same config. Make sure to enter the following commands on both
Spine-1 and Spine-2 switches.
NOTE: Both spine switches will be configured exactly the same. Since we already configured the PIM Feature
and the Loopback 3 Interface, it is rather trivial to enable Anycast RP. The first two lines shown below are all it
takes. The first IP is the IP Address of the RP and the second IP is the loopback0 interface of all the Spine
switches acting as an Anycast RP including this one. The configuration can be copied and pasted on both
spines. The second section is where the RP is statically assigned to the switch
Spine-1, Spine-2:
Spine-1#configure
feature pim
ip pim anycast-rp 10.255.255.255 10.0.0.1
ip pim anycast-rp 10.255.255.255 10.0.0.2
ip pim rp-address 10.255.255.255
end
copy run start
The Leaf switch configuration is simpler than the Spine configuration. Each of the loopback and physical
interfaces that are connected to the Spine Switches must be configured with “ip pim sparse-mode’. The only
other requirement is to specify the Anycast RP address of the Spines.
Lab Guide
Cisco dCloud
Configure RP Address
1. Here, all four Leaf switches will utilize the same config. Make sure to put the config on all Leaf switches. All
of the leaf configurations will use the same IP Address as the RP. A switch will choose either path to the
10.255.255.255 IP Address via uplink. It will not matter which one it picks because they are synchronized
using anycast-rp. The same one line should be applied to each Leaf switch.
Leaf-1, Leaf-2, Leaf-3, Leaf-4:

Leaf-1# configure
feature pim
ip pim rp-address 10.255.255.255
end
copy run start
Configure Interfaces
1. Here, the Spine switches will utilize the same config. Make sure to put the following configuration on both
Spine-1 and Spine-2. Both spine switches will be configured exactly the same. Since we already
configured the PIM Feature and the Loopback 3 Interface, it is rather trivial to enable Anycast RP. The first
two lines shown below are all it takes. The first IP is the IP Address of the RP and the second IP is the
loopback0 interface of all the Spine switches acting as an Anycast RP including this one. The configuration
can be copied and pasted on both spines. The second section is where the RP is statically assigned to the
switch.
Spine-1, Spine-2:
Spine-1# configure
interface loopback 0
ip pim sparse-mode
ip pim sparse-mode
ip pim sparse-mode
exit
ip pim sparse-mode
end
copy run start
Lab Guide
Cisco dCloud
Configure RP Address
1. Here, all of the leaf switches will utilize the same config. Make sure to enter the following configuration on
all four Leaf switches. All of the leaf configurations will use the same IP Address as the RP. A switch will
choose either path to the 10.255.255.255 IP Address via uplink. It will not matter which one it picks
because they are synchronized using anycast-rp. The same one line should be applied to each Leaf Switch.
Leaf-1, Leaf-2, Leaf-3, Leaf-4:

Leaf-1# configure
interface loopback0
ip pim sparse-mode
interface loopback1
ip pim sparse-mode
exit
int ethernet1/1-2
ip pim sparse-mode
end
copy run start
Verification
1. Please be sure to run the following verification commands on both Spine-1 and Spine-2. The output should
be very similar for both switches. However, covering the details of Multicast is outside the scope of this lab.
Spine-1# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD ECMP
Redirect
Priority Capable State Capable
10.0.0.11 Ethernet1/1 16:51:03 00:01:23 1 yes n/a no
Spine-1# show ip pim interface brief
PIM Interface Status for VRF "default"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Ethernet1/1 10.0.0.1 10.0.0.11 1 no
Ethernet1/2 10.0.0.1 10.0.0.12 1 no
Ethernet1/3 10.0.0.1 10.0.0.13 1 no
Ethernet1/4 10.0.0.1 10.0.0.14 1 no
loopback0 10.0.0.1 10.0.0.1 0 no
loopback1 10.0.1.1 10.0.1.1 0 no
loopback15 10.255.255.255 10.255.255.255 0 no
Spine-1#
Spine-2# show ip pim neighbor

PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD ECMP
Redirect
Priority Capable State Capable
Lab Guide
Cisco dCloud

Spine-2# show ip pim interface brief
PIM Interface Status for VRF "default"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Ethernet1/1 10.0.0.2 10.0.0.11 1 no
Ethernet1/2 10.0.0.2 10.0.0.12 1 no
Ethernet1/3 10.0.0.2 10.0.0.13 1 no
Ethernet1/4 10.0.0.2 10.0.0.14 1 no
loopback0 10.0.0.2 10.0.0.2 0 no
loopback1 10.0.1.2 10.0.1.2 0 no
loopback15 10.255.255.255 10.255.255.255 0 no
Spine-2#
This concludes scenario 1.
Lab Guide
Cisco dCloud
Scenario 2. Configuring the BGP Underlay

Value Proposition: For this implementation, the plan is to use iBGP in order to form BGP peering adjacencies.
iBGP is a great way to separate multiple VXLAN Fabrics for future integration. However, there is one rule of
iBGP that must be overcome first. iBGP does not add anything to the AS-PATH so it cannot use the AS-PATH
as a loop prevention option. The loop prevention option in iBGP states that it will not advertise any route it
receives from an iBGP peer to another iBGP peer.
The best way to get around this is to use route-reflectors. Route-reflectors by default do not update the
NEXT_HOP. That is perfectly fine in our scenario as long as the route-reflectors are configured on both spine
switches.
It is also worth noting the additional address-family that might be unfamiliar. It is this new address family that
carries all the control-plane information for EVPN.
Configure BGP Basics on each Switch
On each switch we will enable the necessary two features: BGP and NV Overlay and an enable of Address
family EVPN.
• BGP – The primary protocol for passing reachability information for hosts connected to the VXLAN Fabric
• NV Overlay – Enable the Network Virtualization capability
The “nv overlay evpn” command enables the l2vpn evpn address-family for BGP. The command provides the
capability of using the control-plan for endpoint learning instead of the dataplane with flood and learn.
1. On each Spine and Leaf switch, enter the following commands to enable NV Overlay.
Spine 1, Spine 2, Leaf-1, Leaf-2, Leaf-3 and Leaf-4

Spine-1# configure
feature bgp
feature nv overlay
nv overlay evpn
end
copy run start
2. Enter the following commands to configure the BGP Process. On the Spine switches, the “retain route-
target all” command is required since the Spine switches will be passing the VXLAN traffic, but will not
know about any of the Tenant information. For the most part, each switch will actually be configured the
same. However, it is recommended to specify a router-id that matches the Loopback0 interface.
Spine-1:
Spine-1# configure
router bgp 65001
router-id 10.0.0.1
address-family ipv4 unicast
address-family l2vpn evpn
retain route-target all
end
copy run start
Lab Guide
Cisco dCloud
Spine-2:
Spine-2# configure
router bgp 65001
router-id 10.0.0.2
retain route-target all
end
copy run start
Leaf-1:
Leaf-1# configure
router bgp 65001
router-id 10.0.0.11
end
copy run start
Leaf-2:
Leaf-2# configure
router bgp 65001
router-id 10.0.0.12
end
copy run start
Leaf-3:
Leaf-3# configure
router bgp 65001
router-id 10.0.0.13
end
copy run start
Leaf-4:
Leaf-4# configure
router bgp 65001
router-id 10.0.0.14
end
copy run start
Lab Guide
Cisco dCloud
3. Enter the following commands to configure Spine switches to connect to Leaf switches. Templates are used
to make the configuration more scalable and easier to read. While not completely necessary, it makes the
config cleaner if more neighbors are added.
Spine-1 and Spine-2:

Spine-1# configure
router bgp 65001
template peer iBGP-Leafs
remote-as 65001
update-source loopback0
send-community both
route-reflector-client
send-community both
route-reflector-client
exit
exit
neighbor 10.0.0.11
description Leaf-1 Loopback0
inherit peer iBGP-Leafs
neighbor 10.0.0.12
neighbor 10.0.0.13
neighbor 10.0.0.14
end
copy run start
Leaf-1, Leaf-2, Leaf-3, and Leaf-4:

Leaf-1# configure
router bgp 65001
template peer iBGP-Spines
remote-as 65001
update-source loopback0
send-community both
send-community both
neighbor 10.0.0.1
inherit peer iBGP-Spines
description Spine-1 Loopback0
neighbor 10.0.0.2
inherit peer iBGP-Spines
description Spine-2 Loopback0
end
copy run start
Lab Guide
Cisco dCloud
Verification
4. Enter the following commands on Spine-1 to verify the BGP neighbor relationships formed between the
Spine and Leaf pairs. It doesn’t matter at this time that the tables are empty with 0 routes.
Spine-1# show bgp ipv4 unicast summary
BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 10.0.0.1, local AS number 65001
BGP table version is 6, IPv4 Unicast config peers 4, capable peers 4
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

10.0.0.11 4 65001 9 9 6 0 0 00:00:21 0
10.0.0.12 4 65001 9 9 6 0 0 00:00:19 0
10.0.0.13 4 65001 9 9 6 0 0 00:00:24 0
10.0.0.14 4 65001 9 9 6 0 0 00:00:20 0
Spine-1# show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP table version is 6, L2VPN EVPN config peers 4, capable peers 4

10.0.0.11 4 65001 9 9 6 0 0 00:00:28 0
10.0.0.12 4 65001 9 9 6 0 0 00:00:26 0
10.0.0.13 4 65001 9 9 6 0 0 00:00:31 0
10.0.0.14 4 65001 9 9 6 0 0 00:00:27 0
Lab Guide
Cisco dCloud
Scenario 3. Configuring Overlay

Value Proposition: In this scenario, the goal is to build a VLAN to VNI reference to match the prior Table
information. In reality, it is only 1 addition command that most people aren’t familiar with. The only addition
command is to associate the VNI to the VLAN. This is solved using the “vn-segment” command under a VLAN.
Looking at the design, there are not any hosts plugged into Leaf-4. Leaf-4 is called a “Border Leaf”. It is not
common to plug end hosts into a border leaf. Therefore, the Layer 2 VLAN’s and VNI’s are not necessary to be
configured on the Border Leaf.
Create VLAN/VNI
1. Enter the following commands to configure VLAN/VNI. Note that it is not necessary to configure VLAN 102
on Leaf-1 and Leaf-2 since there are no hosts plugged into it. It is included to make the configuration the
same. Once the lab is completed, it is also recommended to move hosts between VLANs in order to further
enhance understanding.
Leaf-1, Leaf-2, Leaf-3:

Leaf-1# configure
feature vn-segment-vlan-based
vlan 101
name Tenant-1_Network-1
vn-segment 10101
exit
vlan 102
vn-segment 10102
exit
vlan 201
vn-segment 10201
exit
vlan 202
vn-segment 10202
exit
vlan 1001
name Tenant-1_L3VNI
vn-segment 101001
exit
vlan 1002
name Tenant-2_L3VNI
vn-segment 101002
end
copy run start
NOTE: The “Warning” you receive in the command output can be ignored. This occurs due the fact that the
devices are actually Virtual.
Lab Guide
Cisco dCloud
Leaf-4:
Leaf-4# configure
feature vn-segment-vlan-based
vlan 1001
name Tenant-1_L3VNI
vn-segment 101001
exit
vlan 1002
name Tenant-2_L3VNI
vn-segment 101002
end
copy run start
NOTE: Ignore the “Warning” you receive in the command output. This is occurring due the fact that the devices
are actually Virtual.
Create Tenants
1. Enter the following commands on all Leaf switches to create tenants. VRFs are used to separate different
tenants at Layer 3. In reality, it is what makes a fabric multi-tenant. VRFs are not new. However, they do
need to have the proper VNI configured above to match the Tenant and the additional route-target
command for evpn.
Leaf-1, Leaf-2, Leaf-3, Leaf-4

Leaf-1# configure
vrf context Tenant-1
rd auto
vni 101001
route-target both auto
route-target both auto evpn
exit
exit
vrf context Tenant-2
rd auto
vni 101002
route-target both auto evpn
end
copy run start
Build VXLAN Tunnel
At the core, VXLAN is a tunnel. Just like with other tunnel capabilities, it needs a tunnel interface. The NVE
interface is exactly that. It is essentially the VXLAN Tunnel Interface.
In this config, we are setting the “host-reachability protocol” to BGP so that it looks to the BGP Table for all of
its control plane information. It will use the Loopback 1 source interface for the tunnel. The mcast-groups
should be configured to match Table above.
Lab Guide
Cisco dCloud
Leaf-4 is also still a “Border Leaf”. It will require the Layer 3 configurations but none of the Layer 2
configurations.
1. Enter the following commands to build the VXLAN tunnel.

Leaf-1# configure
interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
member vni 10101
mcast-group 239.0.0.101
exit
member vni 10102
exit
member vni 10201
exit
member vni 10202
exit
member vni 101001 associate-vrf
exit
end
copy run start
Leaf-4:
Leaf-4# configure
interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
exit
end
copy run start
Lab Guide
Cisco dCloud
Verification
1. Enter the following commands on Leaf-1 to verify the configuration.

Leaf-1# configure
Enter configuration commands, one per line. End with CNTL/Z.
Leaf-1(config)# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags

--------- -------- ----------------- ----- ---- ------------------ -----
nve1 10101 239.0.0.101 Up CP L2 [101]
nve1 10102 239.0.0.102 Up CP L2 [102]
nve1 10201 239.0.0.201 Up CP L2 [201]
nve1 10202 239.0.0.202 Up CP L2 [202]
nve1 101001 n/a Up CP L3 [Tenant-1]
nve1 101002 n/a Up CP L3 [Tenant-2]
Leaf-1(config)# exit
Configure EVPN
The EVPN section is what sets up Layer 2 connectivity across the fabric. It only requires configuration on Leaf
switches with hosts connected.
1. Enter the following commands to configure EVPN.

Leaf-1# configure
evpn
vni 10101 l2
rd auto
exit
vni 10101 l2
rd auto
exit
vni 10102 l2
rd auto
exit
vni 10201 l2
rd auto
exit
vni 10202 l2
rd auto
end
copy run start
Lab Guide
Cisco dCloud
Configure BGP for Tenant Routing
1. Enter the following commands to add the Layer 3 routing capability across the fabric. Adding the VRF
information to BGP puts all the data into BGP l2vpn evpn table. Utilizing a route-map will bring all the SVI
interface subnets into the BGP process as well.
Leaf-1, Leaf-2, Leaf-3, Leaf-4

Leaf-1# configure
route-map DIRECT permit 10
match tag 12345
route-map DIRECT deny 90
exit
router bgp 65001
vrf Tenant-1
redistribute direct route-map DIRECT
exit
exit
vrf Tenant-2
redistribute direct route-map DIRECT
end
copy run start
Configure SVIs
The SVIs utilize what is called an “Anycast gateway.” This feature puts the same IP/MAC address on each of
the Leaf switches for hosts to connect to. It specifies a “universal” MAC Address for all the switches to use so
that if a host migrates between switches, the gateway MAC address doesn’t change.
1. Enter the following command to configure the feature on all Leaf switches.
Leaf-1, Leaf-2, Leaf-3, and Leaf-4

Leaf-1# configure
feature interface-vlan
end
2. The first SVIs to configure are the Layer 2 SVIs. Notice how they are “tagging” the routes. This will be useful
later when the networks are configured to be routed externally.
Layer 2 SVIs on Leaf-1, Leaf-2, Leaf-3

Leaf-1# configure
fabric forwarding anycast-gateway-mac 1234.1234.1234
interface vlan 101
vrf member Tenant-1
ip address 192.168.101.1/24 tag 12345
mtu 9216
no ip redirects
fabric forwarding mode anycast-gateway
no shutdown
interface vlan 102
vrf member Tenant-1
Lab Guide
Cisco dCloud
ip address 192.168.102.1/24 tag 12345

mtu 9216
no ip redirects
no shutdown
interface vlan 201
vrf member Tenant-2
ip address 192.168.201.1/24 tag 12345
no ip redirects
mtu 9216
no shut
interface vlan 202
vrf member Tenant-2
ip address 192.168.202.1/24 tag 12345
no ip redirects
mtu 9216
no shutdown
end
copy run start
3. Layer 3 SVIs are used to route traffic across the fabric. The configuration is similar to Layer 2 SVIs except
they don’t have an IP address. They use “ip forward” to inform the SVI of its role.
Layer 3 SVIs on Leaf-1, Leaf-2, Leaf-3, Leaf-4
Leaf-1# configure
interface vlan 1001
vrf member Tenant-1
ip forward
mtu 9216
no ip redirects
no shut
exit
interface vlan 1002
vrf member Tenant-2
ip forward
mtu 9216
no ip redirects
no shut
end
copy run start
Lab Guide
Cisco dCloud
Scenario 4. Configure Host Connections

Value Proposition: In this scenario, we will configure Server-2 and Server-3. It is the same configuration that
has been used for years with normal Ethernet switches. The goal here is to verify that the servers can ping their
gateway and each other.
Configure Server-2 and Server-3 Access
1. On Leaf-3, enter the following commands to configure Server-2 and Server-3 access:
Leaf-3:
Leaf-3# configure
int ethernet1/3
switchport
switchport mode access
switchport access vlan 101
spanning-tree port type edge
int ethernet1/4
switchport
end
copy run start
Verification
1. On Server-2, enter the following command to verify the configuration.

cisco@server-2:~$ ping 192.168.101.1
PING 192.168.101.1 (192.168.101.1): 56 data bytes
64 bytes from 192.168.101.1: seq=1 ttl=255 time=5.086 ms
^C
--- 192.168.101.1 ping statistics ---
3 packets transmitted, 2 packets received, 33% packet loss
round-trip min/avg/max = 3.703/4.394/5.086 ms
cisco@server-2:~$ ping 192.168.102.30
PING 192.168.102.30 (192.168.102.30): 56 data bytes
^C
--- 192.168.102.30 ping statistics ---
cisco@server-2:~$
Lab Guide
Cisco dCloud
Configure Server-1 Access (including vPC Configuration)
1. Before the host ports can be configured, the vPC domain must be built. The end goal here is to create a
vPC to Server-1. The end goal is to show redundancy and how to configure it.
Leaf-1:
Leaf-1# configure
feature vpc
feature lacp
vrf context vpc-pka
exit
exit
interface ethernet1/5
no switchport
vrf member vpc-pka
Warning: Deleted all L3 config on interface Ethernet1/5
ip address 192.168.0.0/31
no shutdown
vpc domain 10
peer-keepalive destination 192.168.0.1 source 192.168.0.0 vrf vpc-pka
peer-switch
peer-gateway
ip arp synchronize
exit
interface ethernet1/6-7
switch
switch mode trunk
channel-group 100 mode active
no shutdown
exit
interface port-channel 100
vpc peer-link
end
copy run start
Lab Guide
Cisco dCloud
Leaf-2:
Leaf-2# configure
feature vpc
feature lacp
vrf context vpc-pka
exit
exit
no switchport
vrf member vpc-pka
Warning: Deleted all L3 config on interface Ethernet1/5
ip address 192.168.0.1/31
no shutdown
vpc domain 10
peer-keepalive destination 192.168.0.0 source 192.168.0.1 vrf vpc-pka
peer-switch
peer-gateway
ip arp sync
ip arp synchronize
exit
switch
switch mode trunk
no shut
interface po100
interface port-channel 100
vpc peer-link
end
copy run start
Verification
1. Enter the following command on Leaf-1 to verify the configuration.

Leaf-1# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 0
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
Delay-restore status : Timer is off.(timeout = 30s)
Lab Guide
Cisco dCloud
Delay-restore SVI status : Timer is off.(timeout = 10s)

Operational Layer3 Peer-router : Disabled
Virtual-peerlink mode : Disabled
vPC Peer-link status

---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po100 up 1,101-102,201-202,1001-1002
Leaf-1#
Configure Server-1 Ports
1. Enter the following commands to configure Server-1 ports.
Leaf-1, Leaf-2:
Leaf-1# configure
switchport
no shutdown
exit
interface port-channel101
vpc
end
copy run start
Verification
1. Enter the following commands to verify the configuration.

Leaf-1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
b - BFD Session Wait
S - Switched R - Routed
U - Up (port-channel)
p - Up in delay-lacp mode (member)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
100 Po100(SU) Eth LACP Eth1/6(P) Eth1/7(P)
101 Po101(SU) Eth LACP Eth1/3(P)
Leaf-1# show vpc 101
vPC status
Lab Guide
Cisco dCloud
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
101 Po101 up success success 101
Please check "show vpc consistency-parameters vpc <vpc-num>" for the

consistency reason of down vpc and for type-2 consistency reasons for
any vpc.
Leaf-1#
ubuntu@server-1:~$ ping 192.168.101.1

PING 192.168.101.1 (192.168.101.1) 56(84) bytes of data.
64 bytes from 192.168.101.1: icmp_seq=2 ttl=255 time=4.13 ms
^C
--- 192.168.101.1 ping statistics ---
3 packets transmitted, 2 received, 33.3333% packet loss, time 2003ms
rtt min/avg/max/mdev = 4.134/4.192/4.250/0.058 ms
^C
--- 192.168.101.20 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
^C
--- 192.168.102.30 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
ubuntu@server-1:~$
Lab Guide
Cisco dCloud
Scenario 5: External Routing

Value Proposition: In this scenario, you will set up external routing.
Set up Interfaces on Leaf-4 to reach the WAN Router
1. Enter the following commands to set up the interfaces on Leaf-4 to reach the WAN router.
Leaf-4:
Leaf-4# configure
int ethernet1/3
no switchport
no shutdown
exit
int ethernet1/3.10
encapsulation dot1q 10
vrf member Tenant-1
ip address 172.16.1.0/31
no shutdown
exit
int ethernet1/3.20
encapsulation dot1q 20
vrf member Tenant-2
ip address 172.16.2.0/31
no shutdown
exit
end
copy run start
NOTE: You can ignore the warnings you receive.
Verification
1. Enter the following command to verify the configuration.

Leaf-4# ping 172.16.1.1 vrf Tenant-1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
--- 172.16.1.1 ping statistics ---

5 packets transmitted, 5 packets received, 0.00% packet loss
Leaf-4# ping 172.16.2.1 vrf Tenant-2
PING 172.16.2.1 (172.16.2.1): 56 data bytes
Lab Guide
Cisco dCloud

--- 172.16.2.1 ping statistics ---

5 packets transmitted, 5 packets received, 0.00% packet loss
Leaf-4#
Configure BGP Peering To WAN Router
1. Enter the following commands to configure BGP peering to the WAN router.
Leaf-4:
Leaf-4# configure
router bgp 65001
vrf Tenant-1
neighbor 172.16.1.1
remote-as 65002
exit
exit
exit
vrf Tenant-2
neighbor 172.16.2.1
remote-as 65002
end
copy run start
Verification on Leaf-4
1. Enter the following commands on Leaf-4 to verify the configuration.

Leaf-4# show bgp vrf Tenant-1 ipv4 unicast summary
BGP summary information for VRF Tenant-1, address family IPv4 Unicast

172.16.1.1 4 65002 8 5 65 0 0 00:01:08 1
Leaf-4# show bgp vrf Tenant-2 ipv4 unicast summary

BGP summary information for VRF Tenant-2, address family IPv4 Unicast

Lab Guide
Cisco dCloud
172.16.2.1 4 65002 9 4 52 0 0 00:01:00 1
Leaf-4# show bgp vrf Tenant-1 ipv4 unicast neighbors 172.16.1.1 routes
Peer 172.16.1.1 routes for address family IPv4 Unicast:
BGP table version is 65, Local Router ID is 172.16.1.0
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2
Network Next Hop Metric LocPrf Weight Path

*>e172.16.3.0/24 172.16.1.1 0 0 65002 i
Leaf-4# show bgp vrf Tenant-2 ipv4 unicast neighbors 172.16.2.1 routes
Peer 172.16.2.1 routes for address family IPv4 Unicast:
BGP table version is 52, Local Router ID is 172.16.2.0
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2
Network Next Hop Metric LocPrf Weight Path

*>e172.16.3.0/24 172.16.2.1 0 0 65002 i
Leaf-4#
Configure BGP to Filter Host Routes to WAN Router
1. Enter the following commands to configure BGP to filter host routes to the WAN router.
Leaf-4:
Leaf-4# configure
ip prefix-list NOHOSTS seq 5 permit 0.0.0.0/0 le 31
route-map EBGP-PEER permit 5
match ip address prefix-list NOHOSTS
route-map EBGP-PEER deny 90
exit
router bgp 65001
vrf Tenant-1
neighbor 172.16.1.1
route-map EBGP-PEER out
exit
exit
exit
vrf Tenant-2
neighbor 172.16.2.1
route-map EBGP-PEER out
end
copy run start
Lab Guide
Cisco dCloud
Verification on Server-4
1. Enter the following commands to verify the configuration.

cisco@server-4:~$ ping -c 3 192.168.101.10
PING 192.168.101.10 (192.168.101.10): 56 data bytes
--- 192.168.101.10 ping statistics ---


PING 192.168.101.20 (192.168.101.20): 56 data bytes
--- 192.168.101.20 ping statistics ---


PING 192.168.102.30 (192.168.102.30): 56 data bytes
--- 192.168.102.30 ping statistics ---

cisco@server-4:~$
Lab Guide
Cisco dCloud
Appendix A. Device Toubleshooting

On occasion, a Nexus 9K device may crash due to the Nexus 9000v’s highly demanding IO and available
dCloud environment resources.
For this reason, we have implemented an Out-of-Band method of accessing the serial consoles for the Nexus
9K devices using a guacamole webapp running in a server inside the session called web-consoles.
Google Chrome on the remote desktop is configured to open up the web-consoles webapp and all lab
devices’ serial ports that have been configured within the guacamole application.
If a device has crashed in your session, use the following procedure to recover the failed node.
1. first click the connection of the device name that crashed (for example leaf1_cli).
Lab Guide
Cisco dCloud
2. This places you in rommon mode, where you will see the Loader > prompt.
3. Once you are in rommon mode (Loader> prompt), enter the boot bootflash:nxos.9.3.6.bin command and
press <Enter>.
4. Wait for the switch to finish booting. Once the device finishes booting, you will see the login prompt for the
device (for example Leaf-1).
5. Log in to the device. You can continue the demonstration where you left off prior to the device crash.
Lab Guide
Cisco dCloud
What’s Next?
Now that you have touched the surface of what VXLAN can do, take a look at some of the videos on Cisco
Live’s Website. Search for sessions “BRKDCN-3378” and “BRKDCN-3040”.
After you have a firm grasp on the technology, we recommend that you automate the building of VXLAN Fabrics
using DCNM. There are several YouTube videos demonstrating this process.

Cisco Vxlan Config v1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco Vxlan Config v1

Uploaded by

Copyright:

Available Formats

Lab Guide

Cisco VXLAN Configuration v1

Last Updated: 23-February-2021

Created by Solutions Readiness Engineers

About This Demonstration

About This Demonstration

About This Solution

Scenario 1. Build the VXLAN Underlay

Scenario 2. Configuring the BGP Underlay

Scenario 3. Configuring Overlay

Scenario 4. Configure Host Connections

Scenario 5: External Routing

Appendix A. Device Toubleshooting

Laptop Cisco AnyConnect®

Router, registered and configured for Cisco

About This Solution

• Uses a Layer 3 network to tranfer Layer 2 frames

• Allows for ECMP through the Spine Switches

• Limits spanning-tree to the host facing Leaf switch ports

Name Description Host Name (FQDN) IP Address Username Password

CML Cisco Modeling Labs cml.dcloud.cisco.com 198.18.133.3 guest C1sco12345

Spine-1 Nexus 9K admin C1sco12345

Spine-2 Nexus 9K admin C1sco12345

Leaf-1 Nexus 9K admin C1sco12345

Leaf-2 Nexus 9K admin C1sco12345

Leaf-3 Nexus 9K admin C1sco12345

Leaf-4 Nexus 9K admin C1sco12345

WAN Router IOSv admin C1sco12345

Server-1 Ubuntu Image cisco cisco

Server-2 TinyCore Linux cisco cisco

Server-3 TinyCore Linux cisco cisco

Server-4 TinyCore Linux cisco cisco

Spine-1 10.0.0.1 10.0.1.1 10.255.255.255

Spine-2 10.0.0.2 10.0.1.2 10.255.255.255

Leaf-1 10.0.0.11 10.0.1.11 10.0.1.100

Leaf-2 10.0.0.12 10.0.1.12 10.0.1.100

Leaf-3 10.0.0.13 10.0.1.13

Leaf-4 10.0.0.14 10.0.1.14

Name VLAN ID VLAN Name VNI Multicast Group SVI IP

Tenant-1 101 Tenant-1_Network-1 10101 239.0.0.101 192.168.101.1/24

Tenant-1 102 Tenant-1_Network-2 10102 239.0.0.102 192.168.102.1/24

Tenant-1 1001 Tenant-1_L3VNI 101001 N/A N/A

Tenant-2 201 Tenant-2_Network-1 10201 239.0.0.201 192.168.201.1/24

Tenant-2 202 Tenant-2_Network-2 10202 239.0.0.202 192.168.202.2/24

Tenant-2 1002 Tenant-2_L3VNI 101002 N/A N/A

Name VLAN IP Address Gateway

Server-1 101 192.168.101.10/24 192.168.101.1

Server-2 101 192.168.101.20/24 192.168.101.1

Server-3 102 192.168.102.30/24 192.168.102.1

Server-4 N/A 172.16.3.40/24 172.16.3.1

• TinyCore Linux - 5.4.3-tinycore

1. Initiate your dCloud session. [Show Me How]

NOTE: It may take up to 10 minutes for your session to become active.

• Workstation 1: 198.18.133.252, Username: administrator, Password: C1sco12345.

1. On the remote windows desktop, double-click the devices folder.

Scenario 1. Build the VXLAN Underlay

Build the Underlay IGP Routing Protocol

Loopback addresses described:

Spine-1 and Spine-2:

Leaf-1, Leaf-2, Leaf-3, and Leaf-4:

Verification of the Underlay IGP

10.0.0.1/32, ubest/mbest: 2/0, attached

Configure Multicast IP Routing with PIM

There are actually 2 ways to solve the problem.