OpenStack and NSH Issues

Purpose
● These slides attempt to highlight the issue with SF’s not
being able to receive VxLAN/NSH when used in an
OpenStack environment.
● The current problems are scoped to the NSH V8 patch
which only does VxLAN/NSH encap/decap in the kernel
for VxLAN port types. The forthcoming Intel NSH patch
will add capabilities that can help solve some issues.
Problem
● OpenStack instantiated VMs use tap ports that will not
carry VxLAN/NSH
● Only VxLAN ports will encap and decap VxLAN/NSH
● Because the current NSH V8 patches
○ VxLAN and NSH headers are added on egress
○ VxLAN and NSH headers are stripped on ingress
but stored in metadata that is only visible within
tables
GBPSFC-ENV Full Topology
GBPSFC-ENV Topology

● vxlan/vxlangpe: vtep ports; vxlangpe ports also encap/decap NSH

● vxlan/vxlangpe tunnels could go over the same network since tun_id would isolate traffic
● eth0/eth1: could be a single port; shown here as two physical devices just for clarity
GBPSFC-ENV Assumptions
● All SF’s and SFF’s have VTEPs capable of sending and
receiving VxLAN//NSH
● All SF’s are OVSDB bridges (at the time it was difficult
to find available NSH-aware SF’s). The SF is highlighted
in red in the previous diagram
● So they have VTEPs capable of encapping and
decapping VxLAN and NSH
OpenStack Topology

● vxlan/vxlangpe: vtep ports; vxlangpe ports also encap/decap NSH

● vxlan/vxlangpe tunnels could go over the same network since tun_id would isolate traffic
● eth0/eth1: coul d be a single port; shown here as two physical devices just for clarity
● br-int could really be br-int->br-tun/br-eth with the vxlangpe on br-tun or br-eth, collapsed for clarity
Issues with OpenStack SF’s
● Nova instantiated SF’s (VM’s) are attached to br-int via
a tap port - not a VxLAN port. The SF is highlighted in
red in the previous diagram
● Cannot pass VxLAN or NSH headers to SF’s since they
are tap ports
● Remember only VxLAN ports have the required encap:
○ VxLAN and NSH headers are added on egress
○ VxLAN and NSH headers are stripped on ingress
but stored in metadata that is only visible within
tables
Solutions: Hybrid VxLAN Tunnel
● “Fake” out a VxLAN tunnel between br-int and br-sfc.
● Add VTEP to br-sfc, but no vtep on br-int.
● The idea was that br-sfc would encap the VxLAN/NSH and the
packet would ingress br-int.
● Packet ingress to br-int would simply look like any other l2-l4
packet and could be forwarded to the tap port - with the
VxLAN/NSH.
● Result: OVS tries to be too smart, it recognizes the VxLAN header
and expects a VTEP, otherwise it drops the packet
● Still looking at other tricks to workaround above limitation
Solutions: Hybrid VxLAN Tunnel - 2
Use SF IP as Tunnel Destination
● Bypass the OVS need for having a VTEP on the bridge IP
● Use the IP of the SF
● Example port config and flows (NSH and flow-based programming not shown for clarity):
○ sudo ovs-vsctl add-port br-int vxbrint -- set Interface vxbrint type=vxlan
options:dst_port=6633 options:remote_ip=10.2.1.1 options:key=10 ofport_request=2
○ sudo ovs-ofctl add-flow br-int
in_port=LOCAL,tcp,tp_dst=80,nw_dst=10.2.1.1,actions=output:2
○ sudo ovs-ofctl add-flow br-int
in_port=LOCAL,udp,tp_dst=6633,nw_dst=10.2.1.1,actions=output:1
● Requires adding a route on the host to route traffic back to the bridge. This is per SF, which
means problems with overlapping IPs. Possibly add another discriminator.
● Requires a way to discriminate the SF for incoming tunnel packets. Possibly configure a tunnel
udp port per SF.
● Use any unused IP and UDP port to map to SF’s: worry about address and port collisions
● Add ARP responder rules to reply with br-int mac
Solutions: DPDK NSH Patch
● New patch adds to the NSH V8 patch
● Adds DPDK userspace options to:
○ Encap/decap VxLAN/NSH
○ Encap/decap L2/NSH
● Still have same limitation on requiring a
VTEP for VxLAN/NSH
● Could use L2/NSH directly to the tap port,
but only if using DPDK
Solutions: Change Port Type from tap to VxLAN

● Nova/libvirt will add the tap port to the br-int

as normal
● ODL changes port type to VxLAN so that the
VxLAN/NSH is encapped and decapped
● Result: Not tested yet
Solutions: NSH Proxy
● NSH Proxy implemented to terminate
VxLAN/NSH
● NSH Proxy maps between NSH and non-
NSH worlds on behalf of the SF’s
● Assumes SF’s are non-NSH aware
Solutions: Add Linux Bridge
Solutions: Add Linux Bridge
● Use a linux bridge to “proxy” the VxLAN
NSH packets from br-sfc.
● Need method to get packets from linux
bridge to br-int
○ Use different udp dest port
○ Modify fdb to send packets via veth
● TBD
Solutions: Modify OVS to pass VxLAN/NSH
over Tap Ports
● Modify the OVS code to do encap and decap
of VxLAN without the need for a VxLAN port
type.
● This would be in lieu of using the Intel NSH
patch that does L2/NSH.
Solutions: MPLS
Solutions: Keith’s Ideas