Teccrs 2014

TECCRS-2014
Cisco SDWAN
Technical Deep-dive
David Klebanov
Sukruth Srikantha
Misbah Rehman
Jean-Marc Barozet
Agenda
Time Topic Presenter
8:30-8:40 Kick-Off / Presenters Intro All
Introduction and Background Jean-Marc Barozet
8:40-10:30 Solution Architecture Overview Jean-Marc Barozet
The Fabric Jean-Marc Barozet
10:30-10:45 Break
Overlay Management Protocol Sukruth Srikantha
10:45-12:45
Policies Sukruth Srikantha
12:45-14:30 Lunch
Security David Klebanov
14:30-16:30 Cloud David Klebanov
Application Quality of Experience David Klebanov
16:30-16:45 Break
Management and Operations Misbah Rehman
16:45-18:35
Customer Deployment Use Case Misbah Rehman
18:35-18:45 Wrap-up All
TECCRS-2014 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
cs.co/ciscolivebot#TECCRS-2014
Introduction and
Background
Opening Comments
• Cisco SDWAN is the name for Cisco’s next generation SDWAN
solution for Enterprise & Service Providers.
• vEdge investment for Innovation – “Thin-edge”
• Cisco SDWAN investments for Innovation and for Integration with
IOS-XE on ISR/ASR/ENCS platforms – “Rich-Services”
• Cisco is making significant investments in SDWAN technology and
integration with SDA and ACI solutions
Note: IWAN 2.x support and roadmap will continue as per customer commitments
Network Transformation
Hardware Centric Software Driven
Manual Automated
Closed Programmable
Reactive Predictive
Network Intent Business Intent
CLOUD & ON-PREM AUTOMATION & SCALE SECURITY & COMPLIANCE ASSURANCE & ANALYTICS
Hosted, delivered, managed Speed, flexible, zero-touch, Segmentation, Users, applications, devices
policy driven threat mitigation
Applications Moving to Not One Cloud, But Many
Devices & Things
DC/Private Cloud
WAN
Campus & Branch Users Internet connectivity
becomes
business critical SaaS
Mobile Users
IaaS
More user, things and applications, everywhere

Cisco SD-WAN
Cloud Delivered WAN with
1 Operational Simplicity and Analytics
End-point flexibility: Cloud Delivered Analytics
• Physical or Virtual
4 • Rich Services or Lite
• Branch, Agg, Cloud
USERS
5 3 Application QOE
Cloud
SD-WAN .… Use-Cases
DC
WAN
L EA RN I N G
DEVICES DNA Center

IaaS Apps
Policy Automation Analytics
I N T EN T C O N T EX T SaaS
Intent- based
Network Infrastructure
vDC
S EC U RI T Y
THINGS
Transport Independent Superior Security aAchitecture:

0 WAN Fabric 2 Cloud based and On-prem
Why Fabric Architectures
• Simple: Single Hop, Input / Output

• Overlay on Any Transport
• Consistent Policy Enforcement Points
• Carry New and Useful Context
• User / Device Identity, Network-wide
• Policy Abstraction at User / Group and Application levels
• Policy at Fabric Edge. Over-the-top.
• Increased Simplicity. Seamless Mobility
• Leverage Automation
Deployed Use Cases - Sample
Critical Applications SLA Bandwidth Augmentation Secure Segmentation

• Each vEdge router vManage • Augment MPLS with vManage • Complete isolation in the control vManage
continuously monitors path Internet bandwidth and data plane
App Aware Routing Policy
performance and adjusts • Create traffic engineering Traffic Engineering Policy • Not all VPNs have to be present
App A path must have: Configuration Templates
forwarding Latency ≤ 150ms policy to steer application (data policy) everywhere Assign interfaces and sub-
• Loss ≤ 2% App A - > MPLS TLOC
Configurable probing traffic • Policies are VPN- aware interfaces to respective VPNs
Jitter ≤ 10ms App B - > Internet TLOC
intervals - Active/Active if no policy
Internet
Remote Site Remote Site Remote Site 1 Data Center
Internet VPN1
Internet
MPLS A VPN1 VPN2
Data Center Data Center
MPLS
App A Path 2
VPN2 VPN3
B
4G LTE MPLS Remote Site 2
VPN3 ge0/2.1 - > VPN1
Path1: 10ms, 0% loss, 5ms jitter App A - > MPLS TLOC ge0/2 - > VPN1 VPN1 ge0/2.2 - > VPN2
ge0/3.2 - > VPN2 ge0/2.1 - > VPN1
Path2: 200ms, 3% loss, 10ms jitter App B - > Internet TLOC ge0/3.2 - > VPN2 ge0/2.3 - > VPN3
ge0/3.3 - > VPN3 VPN2
Path3: 140ms, 1% loss, 10ms jitter
SDWAN Tunnel SDWAN Fabric SDWAN Tunnel SDWAN Fabric
© 2017 Cisco and/ or its affiliates. All rights reserved. Cisco Confidential © 2017 Cisco and/ or its affiliates. All rights reserved. Cisco Confidential © 2017 Cisco and/ or its affiliates. All rights reserved. Cisco Confidential
SDWAN Tunnel SDWAN Fabric
Regional Secure Perimeter Guest WiFi DIA & DCA

• Guest WiFi traffic is segmented • DNS- based security
• Firewall service is advertised into the vManage
VPN of choice from regional hub off. Guest WiFi VPN is not
vManage
• Overrides client DNS settings vManage
• Control (or data) policy is used to Service Insertion Policy carried over the fabric.
steer the traffic of interest from (control policy) Guest WiFi Configuration Templates
• Support both simple DIA and DIA Configure DNS server in
remote site through Firewall App A - > Route (data policy)
through Cloud Security service side VPN and
App B - > FW Service App A - > DIA
activate DPI
DNS
Remote Site Internet Internet Query
VPN0
Data Center Remote Site
A
MPLS
Remote Site
B DIA Internet Internet
VPN1
Regional VPN1
App A - > NH Remote Site, LBL VPN1 A Data Center Data Center
FW Service
App A - > NH DC, LBL VPN1 Hub App B - > NH RegHub, LBL FW
App B - > NH RegHub, LBL FW (OMP) MPLS MPLS
(OMP) VPN2 VPN2 DNS Server - > OpenDNS
Regional SDWAN Tunnel
Firewall App A - > DIA
SDWAN Fabric
© 2017 Cisco and/ or its affiliates. All rights reserved. Cisco Confidential © 2017 Cisco and/ or its affiliates. All rights reserved. Cisco Confidential SDWAN Tunnel SDWAN Fabric © 2017 Cisco and/ or its affiliates. All rights reserved. Cisco Confidential SDWAN Tunnel SDWAN Fabric
Connectivity and Overlay
Over The Top Business VPN Extension over Last Mile

End-to-end SD-WAN MPLS Extension
Hosted
with APP level SLA over last mile
Services
MPLS
MPLS
Internet MSP MPLS

Internet Backbone
Site
Site 4G LTE
4G LTE
Cloud
• Transports Managed by Service Providers • Expand Business VPN service over the last mile
or ISP • MSP may not own the transport
• Over the Top (DYI or MSP)
Protecting Workers Wherever They Are…
Datacenter/
Multi-factor Private Cloud
Authentication 1. SDWAN with
• Firewall
Branch • IPS/IDS
Campus • URL Filtering
2. Cisco Umbrella Cisco

• Router intercepts client DNS queries Umbrella IaaS
• DNS queries are forwarded to Cisco Umbrella
DNS servers either unconditionally or based on
the policies
• Cisco Umbrella enforces security policy
compliance based on DNS resolution
• Cisco Umbrella can act as proxy for application
traffic with full Unified Threat Management Secure Internet
capabilities Gateway Internet/SaaS
Home/Mobile
Cloud Networking (IaaS and SaaS)
One Click Cloud Networking (IaaS) Optimized Access to SaaS
Branch to Public Cloud SD-WAN
Hosted Network
Services Private
(MSP Cloud Platform)
Gateway
Regional
Gateway
Transit VPC
Private IP (owned & managed by
MSP)
Internet Private IP
Secure Cloud
Interconnect
Critical
Internet
0r
NetBond
Direct Internet
Internet Hybrid access to
Non- Access Netbond/Secure
Application VPC
critical (owned & managed by (with SaaS Local Cloud
customer) optimization) Breakout Interconnect
• E2E SD-WAN connectivity to business applications in public cloud • Enabling optimal Cloud OnRamp for optimal user experience
• Transport diversity & app aware routing (PIP & Inet) at branch & • SP provided interconnect
public cloud • Direct peering with SaaS/Cloud providers
• Secure private connection to public cloud
Cisco SD-WAN Product Integration Plan
Phase 1 Phase 2 Phase 3
No Integration Platform Integration Management Integration
Deployment Scenarios
Cisco
vManage vManage
DNA Center
vSmart vSmart
+ SD-WAN
vEdge vEdge Cisco Router vEdge ISR4K SD-WAN

Benefits
Thin Edge with vEdge, Rich End-to-end experience with full

Thin Edge with vEdge Edge with XE-SDWAN Cisco DNA integration
Platform: Platform: Management:

vEdge vEdge, ISR1K/4K, CSR1Kv, ENCS, ASR1K Cloud hosted Cisco DNA Center integrates
Details
• • •
Management: Management: vManage capabilities
• vManage • vManage • Full Cisco DNA Center capabilities
(Assurance, Integrated workflows for SD-
Access and SD-WAN)
Solution Architecture
Overview
Cisco SD-WAN Solution Overview
Applying SDN Principles Onto The Wide Area Network
vManage
APIs
Management/
Orchestration Plane
3rd Party
vAnalytics
Automation
vBond
vSmart Controllers
Control Plane
MPLS 4G
INET
vEdge Routers
Data Plane
Cloud Data Center Campus Branch SOHO
Cisco SD-WAN Solution Elements
Orchestration Plane – vBond
Orchestration Plane
vManage
Cisco vBond
APIs
3rd Party • Orchestrates control and

vAnalytics
Automation management plane
• First point of authentication
vBond
(white-list model)
vSmart Controllers • Distributes list of vSmarts/
vManage to all WAN Edges
routers
MPLS 4G
• Facilitates NAT traversal
INET
WAN Edge Routers • Requires public IP Address
[could sit behind 1:1 NAT]
• Highly resilient
Control Plane – vSmart
Control Plane
vManage
Cisco vSmart
APIs
3rd Party • Facilitates fabric discovery

vAnalytics
Automation • Dissimilates control plane
information between WAN Edges
vBond
• Distributes data plane and app-
vSmart Controllers
aware routing policies to the
WAN Edges routers
• Implements control plane
MPLS 4G
policies, such as service chaining,
INET multi-topology and multi-hop
WAN Edge Routers
• Dramatically reduces control
plane complexity
Data Plane – WAN Edge Routers
Data Plane
Physical/Virtual
vManage Cisco WAN Edge
APIs
• WAN Edge router
3rd Party • Provides secure data plane with
vAnalytics remote WAN Edge routers
Automation
• Establishes secure control plane
vBond with vSmart controllers (OMP)
• Implements data plane and
vSmart Controllers application aware routing policies
• Exports performance statistics
MPLS 4G
• Leverages traditional routing
INET protocols like OSPF, BGP and
WAN Edge Routers VRRP
• Support Zero Touch Deployment
• Physical or Virtual form factor
Management Plane - vManage
Management Plane
vManage
Cisco vManage
APIs
3rd Party • Single pane of glass for Day0,

vAnalytics Day1 and Day2 operations
Automation
• Multitenant with web scale
vBond
• Centralized provisioning
vSmart Controllers • Policies and Templates
• Troubleshooting and Monitoring
4G
• Software upgrades
MPLS
• GUI with RBAC
INET
WAN Edge Routers • Programmatic interfaces (REST,
NETCONF)
1, 2, 3 … Fabric
Steps in Establishing Cisco SD-WAN Fabric
Instantiate Control Establish Control Establish Data

Plane Elements Plane Plane
1 2 3
The Fabric
Deploying Fabric Control Plane
Cloud-Delivered Control
Flexible Deployment Options
Cisco Cloud Ops MSP Ops Team Enterprise IT
Deploy Deploy Deploy
vManage vManage vManage
vSmart vBond vSmart vBond vSmart vBond

Cisco MSP Private
Cloud Cloud Cloud
Controllers Deployment Methodology
On-Premise/SP Hosted Cloud Hosted

vBond vManage vSmart vSmart vBond vManage vSmart vSmart
ESXi or KVM AWS or Azure
VM VM
Physical Server Container Container
From Order to PnP - High Level Overview
CCW Ordering
with SA/VA
Hosted or On Prem ON-PREM PnP Connect
Customer Customer add
instantiates Controller
controllers Profile
vManage
PnP Connect
Device
Connect Devices
template
HOSTED PnP Connect
Cisco Controller
Cisco Commerce Profile
instantiates
Workspace Automatically
controllers
added
Configure Device Template

and Attach to S/N
Customer Devices automatically
Service Provider added to PnP Connect
under SA
Control Profile Defined
vBond, Org Name, Root-
Cert
vBond Deployment
NIC1 NIC0
• Virtual machine
• Separate interfaces for control
and management
VPN0 VPN512
• Separate VPNs for control and
management
ge0/0 eth0
- Zone-based security
Control Management
Interface Interface • Minimal configuration for bring-up
- Connectivity, System IP, Site ID,
ESXi, OpenStack, KVM, AWS, MS Azure
Org-Name, vBond IP (local)
vSmart Deployment
NIC1 NIC0
• Virtual machine or container

and management
VPN0 VPN512
management
eth1 eth0
Control Management
Org-Name, vBond IP
vManage Deployment
NIC1 NIC0
• Virtual machine
and management
VPN0 VPN512
management
eth1 eth0
Control Management
Org-Name, vBond IP
vManage Cluster
• Reasons to deploy a vManage cluster
VPN512 - High availability and redundancy for fault
tolerance
VPN0 - Managing greater than 2000 WAN Edge
routers
- Distributing NMS service loads
• Not for geo-redundancy!
• The vManage cluster consists of at least
three vManage devices
• Dedicated interface in VPN0 for cluster
VPN0 communication
• 1Gb bandwidth between cluster members
• <5ms latency between cluster members
https://techzone.cisco.com/t5/Viptela/vManage-Cluster-Creation-and-Troubleshooting/ta-p/1239794/message-revision/1239794:7
The Fabric
Establishing Control Plane
Control Plane Whitelisting
Administrator
Defined
• Administrator adds controllers in the
Controllers vManage GUI
vManage
x.509
• Automated certificate signing through

DigiCert
x.509 x.509
- Can use Enterprise CA
vBond vSmart • Controllers list is distributed by vManage

to all the controllers
- Controllers’ certificates serial numbers
Controller Whitelist in vManage
Controllers Identity
In Software Signed by DigiCert
• Device Certificate* – Own identity (SHA256)

• DigiCert** Root Chain – Trust for other
Root Chain Device controllers’ certificates
Certificate
• Avnet Root Chain – Trust for vEdge routers’
certificates
Root Chain • Cisco Root Chain – Trust for Cisco routers’
certificates (with SUDI)
• Viptela Root Chain (vManage) – Trust for
Root Chain Root Chain vEdge Cloud routers’ and Cisco routers’
(without SUDI) certificates
In Software Provided by vManage CA * Can use Enterprise CA Certificate

(If cluster, one per-member) ** Can use Enterprise CA Root Chain
vEdge Router Identity
During Manufacturing
TPM
Chip
• Device Certificate – Own identity (SHA1)
Device • DigiCert* Root Chain – Trust for controllers’

Certificate
certificates
RootChain
Root Chain
* Can use Enterprise CA Root Chain. Can be loaded

In Software during ZTP.
vEdge
Cisco Router Identity (with SUDI)
During Manufacturing
SUDI
Chip
Device • DigiCert* Root Chain – Trust for controllers’

Certificate
certificates
Root Chain

In Software during PnP.
Cisco Router
Cisco Router Identity (without SUDI)
Signed by vManage
(If cluster, each member signs)

Device
Certificate(s)
• DigiCert* Root Chain – Trust for controllers’
certificates
Root Chain

In Software with Cloud-Init.
Cisco Router
vEdge Cloud, ISRv, CSR1000v Router Identity
Signed by vManage
(If cluster, each member signs)

Device
Certificate(s)
• DigiCert* Root Chain – Trust for controllers’
certificates
Root Chain

In Software with Cloud-Init.
vEdge Cloud
WAN Edge and Controllers White-List
Administrator
Signed
Defined
WAN Edge List
Controllers
vManage • Administrator defined controllers
• Signed WAN Edge list (white-list)
vBond vSmart
• Distributed by vManage to all the
controllers
WAN Edge
Mutual Trust
WAN Edge, vSmart, vManage to vBond
Validate: Root trust, certificate serial,
org-name
 Certificates are exchanged and mutual
authentication takes place
vBond  vBond validates:
- Root of trust for vSmart, vManage and Edge
- Certificate serial* numbers against
authorized white-list (from vManage)
- Organization name against locally
configured one
vSmart WAN Edge vManage
 vSmarts, vManage and Edge validate:
- Root of trust for vBond
- Organization name against locally
configured one
Validate: Root trust, Validate: Root trust, Validate: Root trust,
org-name org-name org-name * Also OTP/Token in case of WAN Edge Cloud and Cisco non-SUDI routers
Mutual Trust
vSmart to vSmart, vManage to vSmart
Validate: Root trust, certificate serial,
org-name  Certificates are exchanged and mutual
vManage  vSmart validates:

- Trust for other vSmart and vManage
- Certificate serial numbers against authorized
white-list (from vManage)
- Organization name against locally configured
one
vSmart vSmart
 vManage validates:
- Trust for vSmart
- Certificate serial numbers against authorized
white-list (from vManage)
Validate: Root trust, Validate: Root trust, - Organization name against locally configured
certificate serial, certificate serial,
org-name org-name
one
Mutual Trust
WAN Edge to vSmart, vManage
Validate: Root trust, Validate: Root trust,
certificate serial certificate serial
org-name org-name
 Certificates are exchanged and mutual
 vSmart and vManage validate:
vSmart vManage
- Trust for WAN Edge
- WAN Edge Certificate serial numbers
against authorized white-list (from vManage)
- Organization name against locally configured
one
 WAN Edge validates:

- Trust for vSmart and vManage
WAN - Organization name against locally configured
Edge one
- Controllers’ Certificate serial numbers
Validate: Root trust, against authorized white-list (from vManage)
certificate serial,
org-name
Control Plane Sessions - Summary
• Secure Channel to SD-WAN DTLS only

• Permanent
Controllers (vSmart, vBond, vManage • Multiple Sessions
vManage)
vBond
• Single extensible control plane
• Operates over DTLS/TLS vSmart1 vSmart2
authenticated and secured
tunnels
• OMP - between WAN Edge DTLS or TLS DTLS or TLS
routers and vSmart controllers • NETCONF • OMP
• Permanent • Permanent
and between the vSmart • Single Session • 1 session / vSmart / TLOC
controllers
• NETCONF – Provisioning from
vManage IPSec
DTLS Only
• Temporary
WAN Edge WAN Edge
vEdge Control Plane Transport
vBond vSmart vManage

• WAN Edge router will by default try to
establish control connections over all
provisioned transports
• Administrator can control which
transports WAN Edge router uses for
establishing control connections
MPLS INET
DTLS
DTLS/TLS
WAN Edge
Firewalls Ports – DTLS
vBond vSmart UDP vManage
Core0 - 12346
Core1 - 12446 UDP
Core2 - 12546 Core0 - 12346
Core3 - 12646 Core1 - 12446
Core4 - 12746 Core2 - 12546
Core5 - 12846 Core3 - 12646
vBond orchestrators do not Core6 - 12946 Core4 - 12746
support multiple cores. vBond Core7 – 13046 Core5 - 12846
orchestrators always use DTLS 12346 UDP UDP Core6 - 12946
tunnels to establish control UDP Core7 – 13046
connections with other devices,
so they always use UDP. The The vManage NMSs and vSmart controllers
UDP port is 12346 can run on a virtual machine (VM) with up to
eight virtual CPUs (vCPUs). The vCPUs are
designated as Core0 through Core7.
Each core is allocated separate base ports for
Firewall control connections
Red signifies primary protocol or first port used

UDP
Default settings: • vBond IP’s are not Elastic, its recommended to
- No Port Offset 12346 permit UDP/12346 to/from any from the WAN Edge
- DTLS WAN Edge 12366 WAN Edge • WAN Edge’s can port hop to establish a connection,
12386
12406 its recommended to permit all 5 UDP ports inbound
12426
to all WAN Edges
The Fabric
Establishing Data Plane
Data Plane Whitelisting and Identity Trust
WAN Edge List Identity • Administrator uploads digitally signed WAN
(White-List) Trust Edge list in the vManage GUI
Valid - White-list for WAN Edge routers
Invalid - Manual upload or Smart Account sync
Staging
vManage
x.509
• Administrator decides on identity trust

vSmart vBond
- Valid, invalid, staging
x.509 x.509
• WAN Edge list and identity trust are

distributed by vManage to vSmart and vBond
On-Boarding on INET Using Global PnP
NSO
Configure Device Template

1 and attach to UUID
PnP Servers
MPLS INET
DMZ (NAT 1:1) 5

3
The router contacts a DHCP

2 server and receives its IP
address from the server.
On Boarding on MPLS with Static IP
• Supported on SD-WAN XE only
• DHCP is not enabled on CE to PE

link (MPLS transport)
• Upon bootup, SD-WAN XE router
will search bootflash: or usbflash:
for filename ciscosdwan.cfg (case
INET MPLS #cloud-boothook
system sensitive)
personality vedge
device-model vedge-C1111-8PLTEEA
• Config file (which includes basic

host-name SITE1_ISR1K
system-ip 10.10.10.10
site-id 501
organization-name "CustomerXYZ - 12345"
console-baud-rate 9600
vbond 64.1.1.2 port 12346
interface configuration, Root CA,
!
!
! Organization Name, vBond
interface GigabitEthernet0/0/0
no shutdown
ip address 192.168.10.10 255.255.255.0
information, etc.) is fed into the
exit
! PnP process
ip route 0.0.0.0 0.0.0.0 192.168.10.1
WAN Edge • Router has all required information

(XE-SDWAN) to connect to vBond
https://sdwan-docs.cisco.com/Product_Documentation/Getting_Started/Hardware_and_Software_Installation/On-Site_Bootstrap_Process_for_SD-WAN_Devices
On Boarding Universal CPE (uCPE)
MPLS INET
Enterprise Networking Compute Platform
x86 runs Virtualization Layer WAN1 WAN2
VNFM
NFVIS
LAN
Quickly roll out new services and location

Ability to run Cisco and 3rd party VNF on NFVIS
On-Boarding – vEdge Cloud, ISRv
vManage Control and Policy
Network Service Orchestrator (NSO) 2 Elements
Core FP Core FP Get the unclaimed vEdge Cloud

(vBranch) (SDWAN-SITE)
router list from vManage. Get
Bootstrap Configuration file (cloud-
Define SDWAN Service on init config file) which contains
1 ENCS (VNF and Chaining) cloud-config (bootstraps) and
cloud-boothook (day0) sections
5
7
3 Full Registration and
Configuration
6
4
VNFs instantiated and loaded with vEdge Cloud,
ISRv
Bootstrap Configuration cloud-init
file. Chaining of VNFs occurred if Virtual Networks
requested. (ENCS)
Data Plane Establishment
vSmart
vSmarts advertise routes and
SD-WAN fabric encryption keys to WAN
between tunnel Edges in OMP updates
endpoints
IPsec Routes and encryption keys
IPsec are advertised to vSmarts in
WAN Edge
IPsec OMP updates
Local Routes
- Local prefixes (OSPF/BGP)
MPLS INET - SD-WAN tunnel endpoints (TLOCs)
Security Context
WAN Edge WAN Edge
- IPSec Encryption Keys
Fabric Routing:
<prefix> via
WAN Edge WAN Edge
Transport Locator (TLOC) OMP IPSec Tunnel
Transport Colors
T3 T4 T1 T2
T3 T4
Internet1 T1 T2 Internet
WAN T1 T3 WAN
WAN T1 T3 WAN Edge Edge
T2 T4
Edge Edge
T2 T4
MPLS
T1, T3 – Internet Color T2, T4 – MPLS Color

Internet2
T1, T3 – Internet1 Color T2, T4 – Internet2 Color
T1 T3 T2 T4
T1 T3 T2 T4
T1 T4 T2 T3
T1 T4 T2 T3
Color restrict will prevent attempt to establish IPSec tunnel to TLOCs

with different color
TLOCs, Colors, Site-IDs and Carriers
• TLOC Color used as static identifier for:

- TLOC Interface on WAN Edge router
- Underlay network attachment
• The specific color used is categorized as Private or Public
- Private Colors [mpls, private1-6, metro-ethernet]
- All other colors are public [red, blue,…, public-internet,…]
• Private vs Public color is highly significant
• Color setting applies to:
- WAN Edge to WA Edge Communication
- WAN Edge to Controller Communication
TLOCs, Colors, Site-IDs and Carriers
Public IP/Port Private IP/Port
1 Private color to Private color
IPSec tunnel – BFD session
2 Private color to Public color
3 Public color to Public color
Fabric Operation
OMP Update:
vSmart  Reachability – IP Subnets, TLOCs
OMP
 Security – Encryption Keys
DTLS/TLS Tunnel  Policy – Data/App-route Policies
IPSec Tunnel
OMP OMP
BFD Update Update
Policies
OMP OMP
Update Update
WAN Edge1 WAN Edge 2

Transport1
T1 T3
T3 T4 TLOCs TLOCs T1 T2
T2 T4
Transport2
BGP, OSPF, VPN1 VPN2 VPN1 VPN2 BGP, OSPF,
Connected, Connected,
Static Static
A B C D
Subnets Subnets
Data Plane - Color Influence
• Colors influence the data plane MSP datacenter

endpoint selection to ensure the NAT
most optimal connectivity
• Domain w/o NAT should use Private
MPLS INET
endpoints, with NAT; use Public
Endpoints
• MPLS uses Private Color, Internet
uses Public Color
• Connectivity optimized within and
across domains
WAN Edge WAN Edge WAN Edge WAN Edge
Data Plane Liveliness and Quality
WAN Edge
• Bidirectional Forwarding Detection (BFD)
• Path liveliness and quality measurement
- Up/Down, loss/latency/jitter, IPSec tunnel MTU
• Runs between all WAN Edge routers in the topology
- Inside SD-WAN tunnels
- Across all transports
WAN Edge WAN Edge
- Operates in echo mode
- Automatically invoked at SD-WAN tunnel
establishment
- Cannot be disabled
• Uses hello (up/down) interval, poll (app-aware)

WAN Edge WAN Edge interval and multiplier for detection
- Fully customizable per-WAN Edge, per-transport
End-to-End Segmentation with Multi-Topology
vSmart
Single Tunnel
Route
Tables
A A Full Mesh Hub and Spoke

B B
C C
WAN Edge WAN Edge
Partial Mesh Point to Point
IP UDP ESP LBL Original Packet
• Segment connectivity across fabric w/o • WAN Edge routers maintain per-VPN routing
reliance on underlay transport table for complete control plane separation
Data Plane Privacy and Encryption
 Each WAN Edge advertises its local IPsec vSmart  Can be rapidly rotated
encryption keys as OMP TLOC attributes
 Symmetric encryption keys used
 Encryption keys are per-transport asymmetrically
Encr-Key3 Encr-Key1
OMP OMP
Encr-Key4
Local (generated) Update Update
Encr-Key2
Local (generated)
MPLS
WAN WAN
Internet
Edge Edge
Remote (received) Remote (received)
IP UDP ESP Original Packet

AES256-GCM/CBC
Encrypted Control Plane
Anti-Replay Protection
 Encrypted packets are assigned  Upon receipt of a packet with higher
sequence numbers. WAN Edge routers sequence number than received thus far,
drop packets with duplicate sequence WAN Edge router will advance the sliding
numbers window
- Replayed packet
 Sliding window is COS aware to prevent
 WAN Edge routers drop packets with low priority traffic from “slowing down”
sequence numbers lower than the high priority traffic
minimum number of the sliding window
- Maliciously injected packet
Drop Accept Range Advance Window
Sliding Window
Packet
Sequence
Numbers
Common Data Plane Communication
Per-Session Load Sharing Per-Session Weighted Application Pinning Application Aware Routing
Active/Active Active/Active Active/Standby SLA Compliant
MPLS INET MPLS INET MPLS INET MPLS INET
SLA SLA
Default Device Policy Policy

Configurable Enforced Enforced
Understanding NAT Types (1/2)
Full-Cone Symmetric
Source: Z / 3001 Source: Z / 3001
Dest: B / 90 Dest: B / 90
Port 90 Port 90
Source: A / 2001 Initial Packet Source: A / 2001 Initial Packet

Dest: B / 90 Host B Dest: B / 90 Host B
Port 91 Port 91
Site Site
NAT NAT
Port 2001 Port 2001
Host A Port 90 Host A Port 90
Host C Host C
Port 91 Port 91
NAT Binding NAT Filter NAT Binding NAT Filter
Local Addr / Port <-> External Addr / Port External Address mask Local Addr / Port <-> External Addr / Port External Address mask
A / 2001 <-> Z / 3001 */* A / 2001 <-> Z / 3001 B / 90
Source: https://www.cisco.com/c/en/us/about/press/internet-protocol-journal/back-issues/table-contents-29/anatomy.html
Understanding NAT Types (2/2)
Restricted-Cone NAT Port-Restricted-Cone NAT
Source: Z / 3001 Source: Z / 3001
Dest: B / 90 Dest: B / 90
Port 90 Port 90
Source: A / 2001 Initial Packet Source: A / 2001 Initial Packet

Dest: B / 90 Host B Dest: B / 90 Host B
Port 91 Port 91
Site Site
NAT NAT
Port 2001 Port 2001
Host A Port 90 Host A Port 90
Host C Host C
Port 91 Port 91
NAT Binding NAT Filter NAT Binding NAT Filter
Local Addr / Port <-> External Addr / Port External Address mask Local Addr / Port <-> External Addr / Port External Address mask
A / 2001 <-> Z / 3001 B/* A / 2001 <-> Z / 3001 * / 90
Source: https://www.cisco.com/c/en/us/about/press/internet-protocol-journal/back-issues/table-contents-29/anatomy.html
NAT Traversal Combinations
Side A Side B IPSec Tunnel Status
Public Public
Full Cone Full Cone
Full Cone Port/Address Restricted
Port/Address Restricted Port/Address Restricted
Public Symmetric
Full Cone Symmetric
Symmetric Port/Address Restricted
Symmetric Symmetric
Direct IPSec Tunnel No Direct IPSec Tunnel (traffic traverses hub) Mostly Encountered
NAT Traversal – Dual Sided Full Cone
vBond
NAT Detection
IP1’ IP2’
Port1 Port2 • vBond discovers post-NAT
vSmart public IP and communicates
back to vEdges
- STUN Server
NAT Filter: NAT Filter:
Any source IP/Port Any source IP/Port • WAN Edge routers notify
IP1’ Full Full IP2’ vSmart of their post-NAT
Port1 Cone Cone Port2 public IP address
• NAT devices enforce no filter
IP1 IP2’ IP1’ IP2 - Full-cone NAT
Port1 Port2 Port1 Port2
WAN Edge WAN Edge
Successful IPSec connection
NAT Traversal – Full Cone and Symmetric
NAT Detection
vBond • vBond discovers post-NAT public
IP and communicates back to WAN
IP1’ IP2’ Edge routers
Port1 Port2
- STUN Server
vSmart
• WAN Edge routers notify vSmart of
their post-NAT public IP address
NAT Filter: • Symmetric NAT devices enforce
NAT Filter: Only from vBond filter
Any source IP/Port From IP1’/Port1
- Only allows traffic from vBond
IP1’ Full IP2’
Port1 Cone
Symmetric
Port2 • WAN Edge behind symmetric NAT
reaches out to remote WAN Edge
- NAT entry created with filter to allow
IP1 IP2’ IP1’ IP2 remote WAN Edge return traffic
Port1 Port2 Port1 Port2 - Remote WAN Edge will learnt new
WAN Edge WAN Edge symmetric NAT source port (data
plane learning)
Successful IPSec connection
Overlay Management
Protocol (OMP)
Overlay Management Protocol Overview
• TCP based extensible control plane protocol
vSmart2
• Runs between WAN Edge routers and
vSmart controllers and between the vSmart
controllers
- Inside permanent TLS/DTLS connections
- Automatically enabled on bring-up
vSmart1 vSmart3
• vSmarts create full mesh of OMP peers
• WAN Edge routers need not peer with all
vSmarts
WAN Edge WAN Edge
OMP Peers
Control Plane Complexity
SD-WAN Traditional IPSec networks
IKE+IPSec
IKE+IPSec IKE+IPSec
OMP OMP
IKE+IPSec IKE+IPSec
IPSec IPSec
IKE+IPSec
Linear Control Plane Complexity Quadratic Control Plane Complexity

O(n) O(n^2)
Overlay Routing: OMP Routes
• Routes learnt from local service
vSmart side
• Advertised to vSmart
controllers
MPLS INET • Most prominent attributes:
OMP Update - TLOC
- Site-ID
- Label
WAN Edge
- Tag
- Preference
Connected
Service - Originator System IP
Static Side - Origin Protocol
- Origin Metric
Dynamic (OSPF/BGP)
- AS PATH
Overlay Routing: TLOC Routes
• Routes connecting locations to
physical networks
vSmart
controllers
OMP Update
MPLS INET • Most prominent attributes:

- Site-ID
- Encap-SPI
TLOCs - Encap-Authentication
WAN Edge - Encap-Encryption
- Public IP
Connected - Public Port
Static
- Private IP
- Private Port
Dynamic (OSPF/BGP) - BFD-Status
- Tag
- Weight
Overlay Routing: Network Service Routes
• Routes for advertised network
vSmart services, i.e. Firewall, IDS, IPS,
generic
controllers
MPLS INET
OMP Update
• Most prominent attributes:
- VPN-ID
WAN Edge - Service-ID
- Originator System IP
- TLOC
Network
Service
Firewall
OMP Best-Path Algorithm and Loop Avoidance
Next hop TLOC is reachable • vSmart will advertise 4 ECMP
paths by default
Prefer Edge-sourced route over vSmart-sourced route - Max 16 paths
• vSmart can send backup path

Prefer OMP route with lower admin distance
for faster reroute on WAN Edge
Prefer OMP route with higher route preference
Prefer OMP route with higher TLOC preference
Prefer highest origin

(Connected, Static, eBGP, OSPF Intra, OSPF Inter, OSPF
External, iBGP, Unknown/Unset)
Prefer route from higher Router-ID (System-IP)
Prefer highest TLOC private IP address
Overlay Routing
• Uniform control plane protocol
Dynamic (OSPF/BGP)
Dynamic (OSPF/BGP) • OMP learns and translates
Static
Static routing information across the
Connected
Connected overlay
Site2
- OMP routes, TLOC routes,
Site1 vSmart network service routes
Overlay - Unicast and multicast address
Management families
Protocol - IPv4 and IPv6 (future)
Site3
Site4 • Distribution of data-plane
Connected security parameters and policies
Connected Static
Static Dynamic (OSPF/BGP) • Implementation of control
Dynamic (OSPF/BGP) (routing) and VPN membership
policies
Policy Framework
Policy Configuration Overview
Policy
Control Data
Affects Control Plane Affects Data Plane
Centralized Localized Centralized Localized

Affects network-wide Route policy in Affects network-wide Access lists
routing site-local network data traffic affects a single interface
on a single router
 Clear separation exists between control plane and data plane policies
 Clear separation exists between centralized and localized functions
vSmart Overlay Policy Architecture
• vSmart Policies consist of these building blocks:
• Lists used for defining targets of policy application or matching
• Policies controlling aspects of control and forwarding
- app-route-policy
- cflowd-template
- control-policy
- data-policy
- vpn-membership-policy
• Policy Application to control towards what a policy is applied
- Site-oriented and defined by a site-list
WAN Edge Service Routing Policy Architecture
• Routing Policies are traditional routing policies
• Attaches to BGP or OSPF locally on the WAN Edge
• Used in the traditional sense for controlling BGP and OSPF
- Information exchange
- Attributes
- Path Selection
• Not covered in detail in this presentation
Policy Framework
vManage
Centralized Control Policy

(Fabric Routing)
Local Control Policy
Centralized Data Policy
(OSPF/BGP)
(Fabric Data Plane) Centralized Localized
Centralized App-Aware Policy Policies Policies Local Data Policy
(Application SLA) (QoS/Mirror/ACL)
VPN Membership
(Fabric Routing+Segmentation)
Centralized Data Policy Centralized App-Aware Policy

vSmart (Fabric Data Plane) (Application SLA) WAN
Edge
Policy Distribution
Data Policy Control Policy
App Aware Routing Policy VPN Membership Policy Local Policies
vManage vManage vManage
NETCONF/YANG NETCONF/YANG NETCONF/YANG
vSmart vSmart vSmart vSmart vSmart vSmart
OMP OMP
WAN WAN WAN

Edge Edge Edge
Building Blocks of Centralized Policies
• Assemble the three building blocks to configure vSmart policies: Groups of Interest,
Policy Definition, and Policy Application.
Groups of Interest Policy Definition Policy Application
Prefixes Control policies affect overlay

routing
Sites An apply directive
TLOC AAR policy with SLAs steer used in conjunction
traffic with site lists enable
VPN specific policies at
Colors specific locations
Data policies provide VPN-level,
SLAs policy-based routing
Centralized policy definition is configured on vManage and enforced

across the entire network
Where Policies are Attached
Direction
Site-ID
vManage Centralized vSmart
Deployment Control Policy
Out
In
Control Policy
Localized
Deployment From Tunnel Direction Data Policy
Site-ID
VPN
WAN Edge
(Site-ID) Data Policy
VPN1 VPN2
From
Service Site-ID
VPN
AAR Policy
LAN1 (from-service only)
LAN2
Order of Operation on WAN Edge
Centralized Local Egress Policy

Routing and Forwarding Access Lists
App-Route Policy Topology-Driven
2 SLA-Based Path Selection 4 Forwarding 6 Policing
Re-marking
Service Side – Transport Side
Local Ingress Policy Centralized Data Policy

1 Policing 3 Policing 5 Queueing and Scheduling
Admission Control Admission Control Shaping
Classification and Marking Classification and Marking and Weighted Round Robin (WRR) with
Re-Marking Low Latency Queuing (LLQ)
Path Selection Congestion Avoidance
Services
Policy Examples
Control Policies
• Configured on vManage. Enabled and enforced on vSmart controllers.
They do not get forwarded to WAN Edge routers.
• Control policies operate on OMP routing information received from or sent to
WAN Edge routers. They can filter OMP updates or modify various attributes.
• Control policies can be very powerful tool changing routing behavior of the
entire SD-WAN fabric
• Control policies are used to enable many services, such as:
- Service Chaining
- Traffic Engineering
- Extranet VPNs
- Service and Path affinity
- Arbitrary VPN Topologies
- and more …
Control Policy – Arbitrary VPN Topologies
• Problem: Different VPNs must be provided with different connectivity based on
applications being serviced in each VPN
VPN 1: CRM System = Hub and Spoke, VPN 2: Voice = Full Mesh
• Solution: Deploy control policy to control VPN topology
Control Policy
Policy Details:
vSmart
VPN1
VPN1 - vSmart advertises just the
Data Center
DC prefixes to Spokes and denies
VPN1 VPN1 everything else on VPN1.
Cisco SD-WAN VPN2 - No filter all the prefixes

are advertised to every node on
Site1 Site3 VPN2
VPN2 Site2 VPN2
VPN1 VPN2
Control Policy – Arbitrary VPN Topologies
policy
lists apply-policy
site-list Branches site-list Branches
site-id 1-3 control-policy ArbitraryTopology out
!
vpn-list CRM
Control Policy
vpn 1
!
vSmart
VPN1
control-policy ArbitraryTopology
Data Center
sequence 10
match route VPN1 VPN1
vpn-list CRM
site-list Branches Cisco SD-WAN
!
action reject
! Site1 Site3
! VPN2 Site2 VPN2
default-action accept
VPN1 VPN2
Control Policy Example – Service Insertion
• Problem: Certain departments require Firewall protection when interacting with data
center networks, while other departments do not
• Solution: Deploy a service chained Firewall service per-VPN
Firewall
Control Policy
Advertise Firewall Service Policy Details:
vSmart Regional Hub
Regional hub advertises
VPN1 - Protected availability of Firewall service
Cisco SD-WAN Bi-directionally modify TLOC

next hop attribute for VPN1
Data traffic between Site1 and Data
Center VPN2 - Open Center to point at regional hub
Site10
TLOCs
VPN1 - Protected VPN2 - Open
! Applied on Regional Hub policy
vpn 1 lists
service netsvc1 address 10.0.1.1 site-list fw-inspected
site-id 10
!
Firewall control-policy fw-service
Control Policy
sequence 10
Advertise Firewall Service
match route
vSmart Regional Hub vpn 1
site-id 1
VPN1 - Protected
action accept
set service netsvc1 vpn 1
Cisco SD-WAN Site1 !
!
Data
VPN2 - Open
Site10 Center apply-policy
site-list fw-inspected
VPN1 - Protected VPN2 - Open control-policy fw-service out
!
! Applied on Regional Hub policy
vpn 1 lists
service netsvc1 address 10.0.1.1 site-list dc
site-id 1
!
Firewall control-policy fw-service-return
Control Policy
sequence 10
Advertise Firewall Service
match route
vSmart Regional Hub vpn 1
site-id 10
VPN1 - Protected
action accept
set service netsvc2 vpn 1
Cisco SD-WAN Site1 !
!
Data
VPN2 - Open
Site10 Center apply-policy
site-list dc
VPN1 - Protected VPN2 - Open control-policy fw-service-return out
!
Control Policy Example – Data Center Priority
• Problem: Prefer main data center over DR data center. If main data center fails, traffic
should reroute to DR data center.
• Solution: Deploy control policy to influence TLOC priority
Control Policy
Policy Details:
vSmart Main DR
DC DC Set higher preference on main
data center TLOCs than on DR
data center TLOCs
Cisco SD-WAN
Preference is set on all TLOC
colors using TLOC list
Site1
Control Policy Example – Data Center Priority
policy
lists
site-list Branches
site-id 3-10
tloc-list Main-DC-tlocs Control Policy
tloc-id 10.1.1.1 biz-internet
tloc-id 10.1.1.1 mpls
vSmart Main DR
control-policy prefer-Main-DC DC DC
sequence 10
match tloc
tloc-list Main-DC-tlocs
action accept
Cisco SD-WAN
set preference 50
apply-policy Site1
site Branches
control-policy prefer-Main-DC out
Control Policy Example – Shared Services
• Problem: Services residing in a VPN must be shared across users residing in multiple
other VPNs. Some VPNs don’t need access to shared services.
• Solution: Deploy control policy with route exports
Control Policy
vSmart
VPN100 Policy Details:
Site2
Export VPN2 and VPN3 routes into
VPN1 shared service VPN100, and vice
versa
Cisco SD-WAN
VPN2 VPN1 cannot communicate with
Site1 VPN2, VPN3 or VPN100
Site3
VPN2 Site4
VPN1 VPN3
Control Policy Example – Shared Services
policy control-policy extranet
lists sequence 10
site-list all-extranet-sites match route
site-id 1-4 vpn-list extranet-clients
vpn-list extranet-clients action accept
vpn-id 2-3 export-to vpn 100
prefix-list extranet-srv-prefix !
ip-prefix 10.1.1.1/32 sequence 20
match route
Control Policy
vpn 100
prefix-list extranet-srv-prefix
vSmart action accept
VPN100 export-to vpn-list extranet-clients
Site2 !
!
VPN1 default-action accept
!
Cisco SD-WAN
apply-policy
VPN2
site-list all-extranet-sites
Site1
Site3 control-policy extranet in
VPN2 Site4 !
VPN1 VPN3
Data Policies
• Data policies are configured on vManage, enabled on vSmart controllers
and enforced on WAN Edge routers
• Data policies allow easier fine-grain traffic controls when compared to
control policies
• Certain objectives can be equally achieved by both control and data
policies. Control policies act on OMP routing advertisements, data policies
act on application traffic characteristics.
• Data policies are used to enable many services, such as:
- Service Chaining
- Cflowd
- NAT
- Traffic Policing and Counting
- Transport Selection, TE
Data Policy Example – Path Preference
• Problem: Send critical applications over MPLS transport and non-critical applications
over Internet transport
• Solution: Deploy data policy to set transport for relevant traffic
Data Policy
vSmart Policy Details:
Bi-directionally set local TLOC for

Site desired traffic
MPLS
Cisco SD-WAN Data Policy Override OMP routing decision

Site
INET Fallback on overlay routing if
Data Policy transport fails
Data Policy Example – Path Preference
apply-policy lists
site-list Site1-2 data-prefix-list DC-Servers
data-policy prefer_mpls from-service ip-prefix 10.1.1.0/24
!
site-list Site1-2
site-id 1-2
Data Policy !
vpn-list vpn10
vSmart vpn 10
data-policy prefer_mpls
vpn-list vpn10
Site sequence 5
MPLS match
destination-data-prefix-list DC-Servers
Cisco SD-WAN Data Policy
source-data-prefix-list Clients
Site !
INET action accept
Data Policy set
local-tloc-list
color mpls
!
Data Policy Example – DIA with NAT
• Problem: Local Internet exit needs to be provided to guest WiFi users. Guest WiFi users
need to be isolated from corporate users.
• Solution: Deploy a data policy in guest VPN with a network address translation
Data Policy
Policy Details:
Internet
vSmart VPN1 – Corporate Define NAT on transport side
interface
DIA NAT Force matching traffic in guest
DIA
Data WiFi VPN through a locally defined
Center VPN2 – Guest NAT on transport side interface
Site NAT
VPN1 – Corporate VPN2 – Guest
Data Policy
Data Policy Example – DIA with NAT
apply-policy
site-list Site1-2
Data Policy data-policy guest-wifi from-service
site-list Site1-2
Internet site-id 1-2
!
vSmart VPN1 – Corporate
vpn-list guest-vpn
vpn 100
DIA NAT policy data-policy guest-wifi
DIA
vpn-list guest-vpn
Data sequence 10
VPN2 – Guest
Site Center action accept
NAT nat use-vpn 0
VPN1 – Corporate VPN2 – Guest !
!
Data Policy default-action drop
!
Application Aware Routing Policies
• Application Aware Routing policies are configured on vManage, enabled on
vSmart controllers and enforced on WAN Edge routers
• Application Aware Routing policies ensure SLA compliant path through the
SD-WAN fabric
• The SLA class defines loss, latency and jitter thresholds
• Application Aware Routing policy matches on the application traffic of
interest. Match can be based on 6-tuple matching or DPI signature.
• Application Aware Routing policy is enforced in VPNs and sites of interest
Application Aware Routing Policy Example
• Problem: Critical applications traffic needs to take SLA compliant path through the
network to achieve better user quality of experience
• Solution: Deploy Application Aware Routing policy for critical application traffic
Application Aware Routing Policy
Critical Application Policy Details:
Site2
Application Aware
Define SLA class for acceptable
Routing Policy SLA thresholds for loss, latency
vSmart
and jitter
Cisco SD-WAN
Non-Critical Application
Apply SLA class to the application
aware routing policy matching on
Site1 the application traffic of interest
Non-Critical Application Critical Application Bi-directionally apply application

SLA Path aware routing policy in the VPNs
Non-SLA Path of choice
Application Aware Routing Policy Example
apply-policy lists
site-list spokes app-list voice
app-route-policy voice-priority app-family audio_video
site-list spokes
site-id 3-5
Application Aware Routing Policy vpn-list vpn10
Critical Application vpn 10
Site2 policy
Application Aware sla-class sla-voice
vSmart
Routing Policy latency 150
loss 1
Cisco SD-WAN !
Non-Critical Application app-route-policy voice-priority
vpn-list vpn10
sequence 1
Site1 match
app-list voice
Non-Critical Application Critical Application !
action
SLA Path sla-class sla-web preferred-color mpls
backup-sla-preferred-color mpls
Non-SLA Path
Policy Definition
Adding a Centralized Policy
• Click Centralized Policy on the Cisco vManage Configuration | Policies screen.
2 If a centralized policy
already exists, you can
1 choose the policy to
modify.
Step1a: Create Groups of Interest
2
1
Step1b: Create Groups of Interest – Prefix Lists
1
3 4
Step1c: Create Groups of Interest – Site Lists
1 3
4
Step1d: Create Groups of Interest – VPN Lists
3
4
Step1e: Create Groups of Interest – TLOC Lists
A TLOC preference influences path

selection.
A higher preference is the preferred path.
The default preference is 0.
Step2a: Define a Topology (Control Policy)
Step2b: Define a Topology – Simple Hub and Spoke
Name and description of the topology
VPN List and Site List are from the groups

of interest previously defined.
Step3a: Configure Traffic Rules (Data Policy)
Step3b: Configure Traffic Rules (Data Policy)
Step4a: Applying Control Policy
Step4b: Applying Data Policy
Activating and Editing Policies
You can only activate one centralized policy at once. Make sure it includes all needed
policies (Control, Data, App-Route, VPN Membership)
Editing
Policies
VPN Membership Policies
• The default behavior of the OMP architecture is to advertise any configured
VPN to any node where it is configured
- Automatically establishes connectivity without unnecessary configuration and
operational overhead
• Certain VPNs may be of a sensitive nature, such that their membership
must be tightly controlled
• The VPN Membership Policy serves to restrict the distribution of VPN
information from vSmart to those that are explicitly approved
- Both Whitelist and Blacklist behavior can be established
• With a VPN Membership Policy, a node not explicitly allowed to participate
in a VPN may have the VPN configured, but will only see local connectivity
and routing information
VPN Membership Policy Example
• Problem: Prevent a site from learning reachability for a VPN, even though this same
VPN is locally defined on the WAN Edge router
• Solution: Deploy VPN membership policy to filter OMP advertisements
VPN Membership Policy

VPN1 Policy Details:
vSmart Site2
VPN1 is defined on Site1 WAN Edge,
however OMP updates pertaining to
VPN1 will not be sent from vSmart to
Cisco SD-WAN Site1 WAN Edge
VPN2
vSmart will not accept any OMP

updates pertaining to VPN1 coming
Site1 from Site1 WAN Edge
VPN1 VPN2
Local Control Policy
• WAN Edge routers can establish standards base routing protocols
adjacencies using OSPF and BGP
• Adjacencies are supported on both service and transport side interfaces
• Adjacencies on the LAN side are used to exchange routing information with
traditional non-SDWAN routers
- Redistribution of OMP overlay routing to OSPF/BGP, redistribution of OSPF/BGP
into OMP
• Adjacencies on the WAN side are used to interact with underlay networks,
when required
• Loop prevention mechanisms are used to prevent routing information
feedback in case of multiple protocol redistribution points, such as
redundant WAN Edge deployment
Local Control Policy Example
• Problem: Dynamically learn remote site prefixes and distribute reachability across
the SD-WAN fabric
• Solution: Enable OSPF dynamic routing on the remote site WAN Edge routers
Policy Details:
vSmart
WAN Edge routers will bi-directionally
Cisco SD-WAN redistribute between OMP and OSPF
OMP OMP
OSPF updates will be sent to site
OMP-to-OSPF OMP-to-OSPF router
OSPF-to-OMP OSPF-to-OMP
OMP updates will be sent to vSmart
WAN OSPF OSPF WAN controller
Edge1 Edge2
Site Router
Local Data Policy
• Local WAN Edge router data policies allow device specific behavior
• Local WAN Edge router data policies cover wide range of functionalities
• Most commonly local data policies are used for:
- Device QoS (queuing, policing, shaping, marking, remarking)
- Local ACLs
- Traffic mirroring
- Deep Packet Inspection
- Flow records
• Local data policies are centrally provisioned through vManage
Local Data Policy – QoS Example
• Problem: Provide differentiated service for various types of application traffic
• Solution: Enable QoS on WAN Edge with proper queuing
vSmart
Map application traffic

into forwarding-classes
Realtime Voice Forwarding Class Q0 20%, LLQ, Tail-Drop
IF
5
Bulk Transfer Forwarding Class Q1 40%, RED-Drop
WAN Edge
QoS Scheduler
cFlowd Policy Example
• Problem: Need to generate application traffic flow records for monitoring and visibility
• Solution: Deploy cFlowd flow export
Flow Collector
Data policy with cFlowd export
VPN1
Policy Details:
Data Center
Define cFlowd template with
vSmart VPN1
export destination IP address
and TCP/UDP port
Include cFlowd export in the
Site2 data policy matching on
VPN2
Site1 application traffic of interest
VPN1 VPN2
Data Policy
Security
Traditional Branch Security
• Security enforcement at the branch is too costly, security
enforcement at the data center is too inefficient (for cloud)
• Segmentation over MPLS is underlay specific, segmentation
over-the-top is operationally cumbersome
Cloud
• Per segment topology… forget about it!
VPN1 VPN2
Users Remote Site
VPN3
Data Center Firewall
Wide Area
Network
Users Remote Site
Layered Branch Security with SD-WAN
• Pick and choose the appropriate security controls
• Embedded DDoS protection
VPN1 Users
VPN2 Compliance
VPN3 DIA and Cloud
SD-WAN Basic Dedicated Cloud

Security Filtering Security Security
SD-WAN Security Overview
Use case: Use Case: Use Case: AMP in 2019
Cloud and DIA Industry Compliance Guest Services
DNS/web
Firewall IPS Firewall URL
vManage Firewall layer security IPS
Filtering
Direct Cloud Access SD-WAN
Cloud VPN1 VPN2 Data Center

Applications Applications
Employee Guest
SD-WAN Security: vManage Provisioning Wizard
Configuration > Security
Application Aware Firewall
• One or more VPNs are mapped to
a zone Internet
• Intra-zone, inter-zone and zone to
DIA traffic policies Inspect policy allows Outside Zone
- Intra-zone and inter-zone traffic only return traffic to
be allowed and drops
between multiple VPNs requires route any new connections
leaking
• 1400+ layer 7 applications WAN Edge
classified
• Block, pass or inspect traffic by Users
Inside Guest
application category or specific Users Zone Zone Devices
application
Service-VPN 1
- Supports 6 tuple matching
Service-VPN 2 Service-VPN 3
Application Aware Firewall Provisioning
Intrusion Prevention and Detection
• Snort IPS engine
Internet
• Runs in a service container on
Cisco ISR4K Routers Signatures
• Backed by global Threat

Intelligence (TALOS) signatures
updated automatically
• Inspects traffic in VPNs of interest
WAN Edge
• Supports three levels of signature
sets
• Signature whitelist support Users Users
• Can run in detection mode
Intrusion Prevention and Detection Provisioning
URL Filtering
• Runs in a service container on Cisco
ISR4K Routers
Internet
• Cloud lookup with local caching or
local lookup
- Local lookup downloads URL database
to the router
• 82+ Web Categories with dynamic
WAN Edge
updates
• Block based on Web Reputation score Users Users
• Create custom Black and White Lists

• Customizable end-user notifications Service-VPN 1 Service-VPN 2
DNS DNS
URL Filtering Provisioning
DNS/Web-Layer Security
Cisco Umbrella
• Cloud-only DNS based inspection
• API Key registration
• VPN-aware policies POP POP POP
• Global points of presence and

snycast IP for fastest response
and high availability
• DNScrypt WAN Edge
• Local domain-bypass
Users Users
• Intelligent Proxy
DNS DNS
DNS/Web-Layer Security Provisioning
Advanced Malware Protection
• File reputation check powered by
Talos AMP
• Sandboxing and file analysis for

unknown signatures powered by Check
ThreatGrid Signature
• Automated signature update from
WAN Edge
ThreatGrid to Talos
Internet
Check file
• Leverages Snort engine to identify
file transfers
ThreatGrid
Malware Sandbox
Advanced Malware Protection Provisioning
SD-WAN Security: Platform Support
App Aware URL DNS/web-

Platforms/Features Firewall IPS
Firewall Filtering layer Security
vEdge 100, 1000, 2000 and 5000) Y Y N/A N/A Y
Cisco CSR1Kv Y Y Y Y Y
Cisco ENCS (ISRv) Y Y Y Y Y
Cisco ISR4K Y Y Y Y Y
Cisco ISR1K (1111X-8P) Y Y Y Y Y

Cisco ASR1K (1001-HX, 1002-HX,
Y Y N/A N/A Y
1001-X, 1002-X)
SD-WAN Security: Platform Requirements
Basic Application Filtering
• Centralized data policy is defined on
vManage vManage and distributed by vSmart
controllers
• Centralized data policy match on
vSmart application traffic of interest
- DPI or 6 tuple matching
Centralized Localized
• Centralized data policy takes drop
Data Policy Data Policy
action to block unwanted traffic
- Can log
WAN WAN
Edge Edge • Localized data policy works similarly
to centralized data policy, but it is
Trust Zone Un-trust Zone distributed directly from vManage
Dedicated Branch Security
Physical Virtual
NIC1 • Inline Firewall to inspect all traffic
WAN
arriving from the LAN environment
Edge
vSwitch1 - Can daisy-chain multiple
WAN services
Edge
• Works for both physical, virtual and

vSwitch2
mixed environments
vSwitch0
x86
• Can be used in conjunction with
Firewall NIC0
SD-WAN security
• Separate Firewall management
Trust Zone Un-trust Zone SD-WAN Security
Dedicated Regional Security
Policy
vSmart Advertisement*
(+ Service) • Service node is connected to
Traffic Path vEdge
Service
OMP Advertisement - Directly or IPSec IKE v1/v2
- Routed or bridged
FW
VPN1 • vEdge router advertises service
VPN1 - Service route + Service label
- Specific VPN
VPN1
Regional • Observe Firewall trust and
Hub/CoLo Data
Center untrust zones
SD-WAN • Control or data policies are used
Remote Fabric to insert the service node
Office
* For data policy only. Control policy enforced on vSmart.
Dedicated Regional Security: Multiple Services
Policy
vSmart Advertisement* • Service nodes are connected to
(+ Service)
vEdge
Traffic Path Service
Advertisement - Directly or IPSec IKE v1/v2
OMP
FW IDS - Routed or bridged
• Service nodes can be connected to
VPN1 different vEdge routers
VPN1
- Can be in different sites
VPN1 • vEdge routers advertise service
Regional
Hub CoLo Data - Service route + Service label
Center - Specific VPN
SD-WAN
Remote Fabric • Control or data policies are used to
Office insert the service nodes
* For data policy only, control policy is enforced on vSmart.
Dedicated DIA/DCA Security
SD-WAN Security for DIA/DCA
• WAN Edge performs DIA
• Port-Address Restricted NAT
Internet
• AppAware Firewall, IPS/IDS, URL-F, AMP
Dedicated Security for DIA/DCA

• WAN Edge performs DIA
• Port-Address Restricted NAT
DIA DIA • Firewall enforces security for DIA traffic
NAT NAT
• Additional protection for WAN Edge
• (Optional) Firewall performs NAT
Site1 Site2 • (Optional) AppAware Firewall, IPS/IDS,
URL-F, AMP on WAN Edge
SD-WAN Security Trust Zone Un-trust Zone
Dedicated Regional DIA/DCA Security
• Internet connectivity is provisioned
in the Regional Hubs/CoLos
Internet
• Regional WAN Edge routers
NAT NAT advertise default route to remote
vSmart
Firewall Firewall
sites’ WAN Edge routers
- VPN aware
• Regional Firewalls provide security
inspection
VPN1 VPN1 • Control policy can constrain
Regional Regional
Hub/CoLo Hub/CoLo default route to a given region
- Region can have multiple hubs for
VPN1
SD-WAN redundancy and load-sharing
VPN1
Fabric
Branch Branch Traffic Path OMP Control Plane
Trust Zone Un-trust Zone
3rd Party Cloud Security
Cloud Security Provider Cloud Security Provider
RGN RGN
POP1 POP2 1 2
DIA ISP A
Regional
Hub/CoLo
ISP B
SD-WAN
Remote Site
Fabric
Remote Site Data Center
GRE/IPSec Tunnels Data Traffic IPSec Tunnels
Application
Quality of Experience
Multidimensional Application Quality of Experience
• Application Visibility and Recognition
• Device QoS
• DSCP/COS Re-Marking
• Application Aware Routing
• Path Remediation
• TCP Optimization
• Fragmentation Avoidance
Application Visibility and Recognition
NBAR2: XE-SDWAN, DPI: vEdge
App 1
App 2
Cloud
Data Center App N
Application Recognition
Application Visibility
Data Center
MPLS 4G
INET
Small Office
Home Office
Campus
Branch
Device QoS: Queuing
• Per-Egress Interface Queuing • Q1-Q7: Weighted Round Robin
- 8 queues - Bandwidth percent determines queue
• Classification weight
- 6-tuple or DPI • Q1-Q7: Queue drop is RED* or tail-drop
- Local or central data policy - Linear drop probability, i.e. X% queue depth
results in X% drop probability
• Q0: Control traffic
- DTLS/TLS, BFD, routing protocols WAN Edge
- Not subjected to LLQ policer Q0
Egress Interface
Ingress Interface
• Q0: LLQ Q1
Q2
- Unused bandwidth is distributed
between Q1-Q7
Q7
* Random Early Discard

Classification Queuing
Device QoS: Shaping
• Egress physical interfaces • Exceeding shaping rate: Queue
- Not supported on sub-interfaces - There are no tokens in the bucket
• Classification - Weighted Round-Robin
- Interface-level
Rate
• Conforming to shaping rate: Forward Tokens
Token Bucket
- There are tokens in the bucket
• Exceeding shaping rate: Queue WAN Edge
- There are no tokens in the bucket
Egress Interface
Ingress Interface
- Weighted Round-Robin
Classification Shaping Queuing

Device QoS: Policing
• Ingress and Egress Policing • Burst Rate: Configurable
- Interface or sub-Interface - Token bucket depth
• Classification
- [Sub] interface, 6 tuple or DPI
Rate
- Local or central data policy Tokens
• Conforming to policing rate: Forward Token Bucket
- There are tokens in the bucket WAN Edge

• Exceeding policing rate: Drop/Remark
Egress Interface
Ingress Interface
• Burst Rate: Configurable
- Token bucket depth
Classification Policing Queuing

Device QoS: Policing with Packet Loss Priority
Rate
Tokens • Set PLP=High value for traffic that
Token Bucket exceeds configured policer rate
- Default is PLP=Low
• Data policy can match on PLP
high value and set different local
TLOC A TLOC
Policing
- Per-packet
- Endhost reconciles out-of-order
Conforming Rate packets
Exceeding Rate • Non-conforming traffic spills over
to a different circuit
TLOC B
DSCP and COS (802.1p) Re-marking
Copy original DSCP markings • Comply with service provider
into outer DSCP markings
provisioned classes of service
• (Optional) Original DSCP rewrite
- Classification: 6 tuple or DPI
Egress - Action: Local or central data policy
Ingress Interface
Interface • (Default) Original DSCP marking is
copied to the outer DSCP marking
802.1p
DSCP
DSCP
DSCP
• (Optional) Egress outer DSCP rewrite

- Re-write rules based on forwarding
class mapping on ingress
Classify: 6 tuple or DPI Modify with • (Optional) Egress COS rewrite
Action: set DSCP, map into re-write rules
forwarding class (FC) (per-FC) - Re-write rules based on forwarding
class mapping on ingress
Path Quality Detection
• Each WAN Edge router initiates BFD
packet every hello interval
- Echo mode, no neighbors
App-Route Multiplier (n) - Tunable to sub-second level
• Poll interval determines the window for
Poll Interval Poll Interval Poll Interval (ms)
calculating path quality
- Averaged
- Tunable to sub-second level
• App-route multiplier determines
Hello Interval (ms) number of poll intervals for
establishing overall average path
quality
- Compared against application aware
routing thresholds
Critical Applications SLA
vManage App Aware Routing Policy
 WAN Edge Routers
App A path must have:
continuously perform path
Latency < 150ms
liveliness and quality Loss < 2%
measurements Jitter < 10ms
Internet
MPLS
Remote Site Path 2 Data Center
4G LTE
SD-WAN IPSec Tunnel
Forward Error Correction (FEC)
• Protects against packet loss • Supports multiple transports
• Protocol (TCP/UDP) agnostic • Can be invoked dynamically
• Operates per-tunnel • Applied with data policy
XOR XOR
Notes:
• Application traffic only, not BFD
1 2 • Parity packet matches the transport and DSCP 1 2
P value of the last packet in the block 3
• Parity packet size is the max size of the
3 4 packet in the block P 4
5 6 1 2
7 8 3 4
Sender Receiver
SD-WAN Tunnel FEC Header

FEC and Multiple Circuits
• Multiple flows may require FEC • Block parity uses multiple flows
• Preserves per-flow hashing • Protects against bursty loss
XOR XOR
Flow1 1 2 Flow1 1 2
P 3
Flow2 3 4 Flow2 P 4
SD-WAN Tunnel
2 1
Block
Sender P 4 3 Receiver

FEC and Application Aware Routing
• Works independently • AppAware chooses SLA tunnel(s)
• AppAware first, data policy next • Data policy applies FEC
XOR XOR
SLA SLA
Flow1 1 2 Flow1 1 2
P 3
Flow2 Flow2
3 4 SD-WAN Tunnel P 4
(SLA Compliant)
2 1
Block
Sender P 4 3 Receiver

Packet Duplication
• Protects against packet loss • Operates over multiple tunnels
• Protocol (TCP/UDP) agnostic • Applied with data policy
Notes:
• Works only over multiple tunnels
1 2 1 2
• Duplicates are discarded on receiver
3 4 3 4
SD-WAN Tunnel
D D
4 3 2 1
D D
Sender 4 3 2 1 Receiver
SD-WAN Tunnel
Packet Duplication and Application Aware Routing
• Works independently • AppAware chooses SLA tunnel(s)
• AppAware first, data policy next • Data Policy applies duplication
Notes:
• Entire application aware policy logic applies
SLA SLA
1 2 • Packets are duplicated to the least lossy 1 2
Flow1 remaining tunnel Flow1
Flow2 3 4 Flow2 3 4
SD-WAN Tunnel
(SLA Compliant)
D D
4 3 2 1
D D
Sender 4 3 2 1 Receiver
SD-WAN Tunnel
TCP Optimization
Optimized
TCP Connections TCP Connections TCP Connections
SD-WAN
Fabric
Users vEdge vEdge Application
Router High Latency / Lossy Path Router Servers
• High latency or/and lossy path between • Optimized TCP connections use selective
users and applications, i.e. geo-distances acknowledgements to prevent
unnecessary retransmissions of received
• vEdge routers terminate TCP sessions
segments
and provide local acknowledgements
- Hosts don’t have to wait for end-to-end • Hosts using older TCP/IP stacks will see
TCP ACKs and pause TCP transmission the most benefit
Optimal MTU with TCP MSS Adjust
MTU MTU
1500 Bytes SD-WAN Fabric 1500 Bytes
IPSec
Automatic Tunnel MTU
Host WAN Edge WAN Edge Application
Discovery using BFD
Router Router Servers
Signaled MSS Signaled MSS

1460B MSS Adjust 1320B Send MSS
to 1320B 1320B
Signaled MSS Signaled MSS
Send MSS 1320B MSS Adjust 1460B
1320B to 1320B
 Send TCP MSS is min (local link IP MTU - 40B, signaled MSS value)
- Signaled in SYN packets
 Can manually set TCP MSS value on WAN Edge router
- Per-interface
Optimal MTU with Host PMTUD
IP MTU 1500 Bytes SD-WAN
Service Side Transport Side Fabric
Host WAN Edge Transport(s)

Network

Discovery using BFD
DF=1 Fragmentation
Host
Packet
1500B Needed
Adjust IP MTU
Inner Outer
Packet
Packet DF=1 No (DF=1) DF=1 No
< 1500B Fragmentation Fragmentation
Packet Fragmentation
IP MTU 1500 Bytes SD-WAN
Service Side Transport Side Fabric
Host WAN Edge Transport(s)

Network

Discovery using BFD
Inner Outer
Host
Packet DF=0 Fragmentation (DF=0) DF=1
1500B Needed
Fragment
 WAN Edge routers perform fragmentation, then encapsulation

 Reassembly is done by the endhosts
Cloud Adoption
Shifts in Enterprise Workloads
IaaS SaaS
Traditional On-Premise Data Centers
Traditional Cloud Applications Access
• Data Center backhaul

• Increased application latency
• Unpredictable user experience
Wide Area
Network
Data Center
Users Remote Site
SD-WAN Cloud Applications Multipathing
Problem:
Which way is cloud?
Regional
Hub/CoLo
SD-WAN
Data Center
Users Remote Site
Cloud Application Access Cloud Application Access

without SLA with SLA
Recreational Browsing
Guest Access Business Critical Applications
Generic Cloud Applications
1 2
Route to Cloud with DIA(s)
Cloud
NAT
VPN0
Remote Site
vSmart
SD-WAN Router
VPN1
NAT VPN0 Default Route or
NAT VPN0 Data Policy Action
Regional • All or policy selected application

NAT NAT Hub/CoLo traffic
INET1 INET2
- Per-VPN
• DIA selection using local TLOC
SD-WAN color
Fabric • Secure Access
Data Center - Port-Address Restricted NAT
Remote Site
 Integrated SD-WAN Security
OMP Application Traffic - NG-Firewall
Route to Cloud with OMP
Cloud
BGP/OSPF/Static
0.0.0.0/0
VPN1
vSmart Hub/CoLo/DC
NAT SD-WAN Router
VPN0
INET
SD-WAN Remote Site
Regional
Hub/CoLo NAT
• OMP routed application traffic
Rcv OMP: - Per-VPN
0.0.0.0/0 INET
• Regionalized or centralized
SD-WAN • Secure Access
Fabric - Port-Address Restricted NAT
Data Center - Integrated SD-WAN Security
Remote Site
 NG-Firewall
OMP Application Traffic
Cloud Application Access Cloud Application Access

without SLA with SLA
Recreational Browsing
Guest Access Business Critical Applications
Generic Cloud Applications
1 2
Cloud onRamp for SaaS – Multiple DIA
Overview
• Detect application performance

through one or more Direct
Internet Access circuits
Loss/ Best
Latency Performing • vEdge routers chose best
! performing path
INET1 INET2 - Per-Application, Per-VPN
• Automatic failover in case of
performance degradation
SD-WAN
Fabric • Fully automated
Remote Site
Quality Probing
Cloud onRamp for SaaS – DIA(s) and Gateway(s)
Overview
• Detect application performance

Best through DIAs and gateways
Performing - Customer/SP owned and
operated
Loss/
Latency
INET2 - Security, performance, reliability
! • vEdge routers chose best

Regional
INET1 Hub/CoLo performing path
- Per-Application, Per-VPN
SD-WAN • Automatic failover in case of

MPLS Fabric performance degradation
Remote Site • Fully automated
Quality Probing
Cloud onRamp for SaaS Operation
Discover Cloud Determine Cloud Route Cloud

Application Application Performance Application Traffic
1 2 3
Cloud onRamp Application Discovery
DNS Server(s)
• DNS server(s) are defined in VPN0

• Network Address Translation is
configured on all DIA interfaces
NAT NAT
• vEdge router performs DNS resolution
INET1 INET2 for the configured Cloud onRamp for
SaaS applications
IF IF - Done separately over each DIA circuit
- Can resolve to different IP addresses
VPN0
vEdge Router
DNS Query
Cloud onRamp Application Probing
• vEdge router initiates periodic HTTP
pings toward the configured Cloud
onRamp SaaS applications
- Done separately over each DIA circuit
NAT NAT
1 2 3 4
INET1 INET2 Probe
1 2 10 Sleep
1s 20s
IF IF 30s
VPN0 Bucket Bucket Bucket Last 6 buckets

#1 #2 #6 determine average
loss and latency
vEdge Router 12min

Quality Probe
Cloud onRamp Application Performance
• vEdge router determines average loss and

latency toward cloud applications based on
HTTP probes measurements
Best Loss/
Performing Latency - vQoE [Loss] = Desired Loss / Average Loss * 100
! - vQoE [Lat] = Desired Lat / Average Lat * 100
INET1 INET2
• vEdge router determines best performing DIA
path based on vQoE score
IF IF
- vQoE = Average (vQoE[Loss] + vQoE[Latency])
0 5 8 10
VPN0
Bad Average Good
vEdge Router
Quality Probe
Cloud onRamp Host DNS Resolution
DNS Server(s)
• Host performs DNS resolution

Best Loss/ • vEdge Router DPI engine intercepts
Performing Latency
host DNS query
!
INET1 INET2 • If host DNS query is for the Cloud
onRamp SaaS applications, vEdge
IF IF Router forwards it to the DNS server
VPN0 defined under VPN0 over best
performing DIA circuit
1 DPI - Overrides user DNS settings
2 SD-WAN
DNS Query • DNS queries for non-Cloud onRamp
Host Intercepted Fabric
vEdge Router applications are forwarded according
to the routing table
DNS Query for Cloud DNS Query for non-Cloud
1 onRamp SaaS application
2 onRamp SaaS application - User DNS settings are preserved
Cloud onRamp Application Steering
• Host initiates communication with the
Cloud onRamp SaaS application (1)
Best Loss/
Performing Latency • vEdge Router may choose sub-performing
NAT1 NAT2
DIA circuit for the initial application flow
INET1 INET2 ! - DPI engine had not yet identified the Cloud
onRamp SaaS application
IF IF • Once vEdge Router DPI engine identifies
Cache Table
dstIP/dstPort -> SaaS App Cloud onRamp SaaS application, cache
VPN0
(INET1 IF) table is populated and all subsequent
1 DPI
application flows are routed over best
Host A
2
performing DIA circuit (2)
Host B 2 vEdge Router
- Overrides routing decision
• Initial application flow is not rerouted, even
1 Initial TCP flow TCP flows over best if using sub-optimal DIA circuit
2
performing path - NAT changes will break TCP flow
Cloud onRamp Application Discovery w/ GW
DNS Server(s)
NAT • DNS server(s) are defined in VPN0

INET2
DNS Server(s) • Network Address Translation is
IF configured on all DIA interfaces
NAT
• vEdge Routers at remote location
VPN0 and gateway perform DNS
INET1
resolution for the configured Cloud
IF
vEdge Router
(gateway)
onRamp for SaaS applications
SD-WAN - Can resolve to different IP
VPN0 Fabric addresses
vEdge Router
(remote site) DNS Query
Cloud onRamp Application Probing w/ GW
• vEdge Routers at remote location
and gateway initiate periodic HTTP
NAT
pings toward the configured Cloud
Best
Performing INET2 onRamp SaaS applications
IF • vEdge Router at the remote site
Loss/ determines best performing path
Latency
NAT
toward the Cloud onRamp SaaS
! VPN0
INET1 applications based on loss and
IF
vEdge Router latency characteristics
(gateway)
- Compares between local DIA(s) and
SD-WAN composite metric of HTTP ping + BFD
VPN0 Fabric through the gateway vEdge Router
vEdge Router
- Gateway probe stats are distributed
(remote site) using OMP through vSmart
HTTP ping BFD
Cloud onRamp Host DNS Resolution w/ GW
DNS Server(s)
• Host performs DNS resolution
Best
DNS Query for Cloud
Performing
• Remote site vEdge Router DPI engine
1
onRamp SaaS application
intercepts host DNS query
DNS Query for non-Cloud
2
onRamp SaaS application INET2 • If local DIA circuit is the best path,
remote site vEdge Router forwards DNS
Loss/ IF
Latency query to the DNS server defined under
VPN0 VPN0 over local DIA circuit
! • If gateway is the best path, remote site
INET1
DPI vEdge Router forwards DNS query to the
IF
DNS Query
Intercepted
gateway vEdge Router, which in turn
vEdge Router
forwards it to the DNS server defined
VPN0
(gateway) under VPN0 over it’s local DIA circuit
1
DPI SD-WAN - Gateway vEdge Router DPI engine intercepts
2 DNS query to make a decision
Host
DNS Query Fabric
Intercepted • DNS queries for non-Cloud onRamp
vEdge Router applications are forwarded according to
(remote site)
DNS Query the routing table
Cloud onRamp Application Steering w/ GW
Best • Host initiates communication with the
Performing
Cloud onRamp SaaS application (1)
1 Initial TCP flow NAT2
• Remote site vEdge Router may choose
2
TCP flows over best INET2 sub-performing path for the initial
performing path Cache Table application flow
dstIP/dstPort -> SaaS App
(INET2 IF)
IF - DPI engine had not yet identified the
Loss/ VPN0 Cloud onRamp SaaS application
Latency NAT1
vEdge
! • Once remote site vEdge Router DPI/
INET1 Router
(gateway)
DPI engine identifies Cloud onRamp SaaS
Cache Table
dstIP/dstPort -> SaaS App application, cache table is populated and
(tunnel to gateway)
IF all subsequent application flows are
VPN0 routed over best performing path
1 SD-WAN - Overrides routing decision
DPI
Host A
2 Fabric • Initial application flow is not rerouted,
vEdge Router
even if using sub-optimal path
Host B 2
(remote site) - NAT changes will break TCP flow
Traditional IaaS Access
• No Direct to Cloud access
• Limited segmentation and QoS
• Dependent on underlying technology
IPsec
IPsec
Data Center IPsec VNET VNET
Azure Express VNET VNET

Route
Wide Area
Network
Remote Site CNF/CoLo
VPC VPC
VPC VPC
AWS Direct
Connect
Cloud onRamp for IaaS: Marketplace DIY
• vEdge Cloud routers are instantiated in

Compute Compute
VPC/VNET VPC/VNET Amazon VPCs or Microsoft Azure VNETs
- Posted in marketplace
- Use Cloud-Init for ZTP
Cloud • One vEdge Cloud router per VPC/VNET
Data Center
- Redundancy is handled in cloud provider
• vEdge Cloud routers join the fabric, all

SD-WAN fabric services are extended to the IaaS
Fabric instances, e.g. multipathing, segmentation
Campus
Remote Site and QoS
- For multipathing, can combine AWS Direct
Connect or Azure ExpressRoute with direct
Internet connectivity
Branch
Cloud onRamp for IaaS: Automated AWS
Standard IPSec + BGP
(2x) SD-WAN • Gateway VPC per-region
VPC
BGP <-> OMP - Multiple for scale
AZ1
• VGW for host VPCs
VPC
AZ2
VGW
• Standard based IPSec
AZ1 INET - Connectivity redundancy
Host VPC WAN Edge
MPLS
• BGP across IPSec tunnels for
AZ2 Direct route advertisement
VPC WAN Edge Connect - Active/active forwarding
AZ1 - BGP into OMP redistribution
Transit VPC
- Advertise default route to host
VGW VPCs
AZ2
• Optional AWS Direct Connect
Host VPC
AWS Region
vManage
Cloud onRamp for IaaS: Automated Azure
Standard IPSec + BGP
(2x) SD-WAN • Gateway VNET per-region
VNET
BGP <-> OMP - Multiple for scale
AS1
• VPN GW for host VNETs
VNET
VPN
AS2 GW • Standard based IPSec
INET - Connectivity redundancy
Host VNET WAN Edge
MPLS
• BGP across IPSec tunnels for
Express route advertisement
VNET WAN Edge
AS
Route - Active/active forwarding
GW
AS1 - BGP into OMP redistribution
VNET Gateway
- Advertise default route to host
VPN
GW VNETs
AS2
• Optional Azure Express Route
Host VNET
Azure Region
vManage
Cloud onRamp for IaaS with Segmentation
• End to end segmentation over SD-WAN fabric
• VPCs map to SD-WAN VPNs
• VPC belongs to a single SD-WAN VPN [ ]
HR
VPN1 VGW
VPC
VPC HR
Resources
VPN2
VPN1
Finance SD-WAN
VPN2 VPC
VPN2 Finance
Transit VPC Resources
VGW
Finance
Cloud onRamp for IaaS with AppAware Routing
• Application Aware Routing over SD-WAN fabric
• Leverages public and private transports
• Also adds resiliency [ ]
IGW VPC
Users INET
IPsec
SD-WAN Resources
CNF/
Users CoLo
Direct Connect Transit VPC
Cloud onRamp for IaaS Scale
New AWS Transit VPC or Azure VPN Gateway when the number of IPsec
tunnels toward the host VPCs or VNETs exceeds maximum supported number
IPsec
Resources
Users
Transit/Gateway
SD-WAN
Users IPsec
Resources
Transit/Gateway
Management and
Operations
Agile Operations
Power Tools
CLI Linux Shell
REST NETCONF Syslog SNMP Flow Export

Controller Tenancy
Single Tenant Multi Tenant
vManage vBond vManage vBond
1 1 1 2 3 1 2 3
VM VM VM VM
VM/ VM/
Container Container
1 1 2 3
vSmart vSmart vSmart vSmart
AWS, MS-Azure, KVM, ESXi
Multi-tenancy
MSX
A B A+B
Dedicated VPN
(No) Tenancy Tenancy
VPN1 MPLS 4G VPN1 VPN1 VPN2

MPLS 4G
VPN2 VPN2
VPN3 INET VPN3 INET
Tenant VPN1 VPN1 Tenant Tenant VPN2 VPN1 Tenant
VPN2 VPN2 A B
A VPN3 VPN3 B
Tenant Tenant A Tenant Tenant

B A A+B B A+B B A
Enterprise
Tenancy
VPN1 MPLS 4G VPN1

VPN2 VPN2
VPN3 INET VPN3
Tenant VPN1 VPN1 Tenant Control Plane

VPN2 VPN2
A VPN3 VPN3 B
Tenant Tenant
B A
Horizontal Solution Scale
Orchestration Plane Management Plane Control Plane
(vBond) (Multi-tenant or Dedicated) (Containers or VMs)
(vManage) (vSmart)
Horizontal Scale Out Model
Add vBond Orchestrators to increase Create vManage cluster to accommodate Add vSmart Controllers for more
WAN Edge bring-up capacity more WAN Edge routers control plane capacity
• Choose WAN Edge platform with

appropriate IPSec tunnel scale
4G/LTE Internet
• Use control policies to define
MPLS VPN topologies
Data Center Campus Branch Home Office
Horizontal Solution Scale – Control Plane
5400
vBond vSmart Con vManage
1500 1500 1500 2000 2000 2000
Con Con Con 5400 5400 Dev Dev Dev
Con Con
x8 x6
x20
FQDN Networked Cluster
Hash
DNS 1 permanent connection
Hash
per-transport
1 permanent connection
1 transient connection
per-transport WAN Edge
High Availability and Redundancy Overview
Site Redundancy Transport Redundancy
MPLS INET MPLS INET
VRRP OSPF/ OSPF/

BGP BGP
Network/Headend Redundancy Control Redundancy

vSmart Controllers
MPLS Control
Data
Center
INET Data MPLS
Site
INET
Redundancy - Site with LAN Routing
• Redundant WAN Edge routers

SD-WAN
Fabric • OSPF/BGP between WAN Edge routers
and site router(s)
• Bi-directional redistribution between OMP
WAN WAN
Edge A Edge B
and OSPF/BGP
- Loop prevention
Site Router Site Router • Multipathing for remote destinations across

SD-WAN Fabric
- Can manipulate OSPF/BGP to prefer one
WAN Edge router over the other
Host
Redundancy - Site with LAN Bridging
SD-WAN • VRRP between WAN Edge routers

- Operates per-VLAN
Fabric
• VRRP Active WAN Edge router responds to
ARP requests for the virtual IP and virtual
A S A S
WAN VRRP Grp 1 WAN MAC*
Edge A Edge B
VRRP Grp 2 • Prior to 18.3.0, in case of failover, new
VLAN 1
VLAN 2
VRRP Active WAN Edge (vEdge) router
sends out gratuitous ARP to update ARP
table on the hosts and mac address table
on the intermediate L2 switches
Host Host
* Virtual MAC requires minimum 18.3.0 code on vEdge
Redundancy – Meshed Transports
• WAN Edge routers are directly • SD-WAN tunnels are built through
connected to all the transports all directly connected transports
Circuit Failure Transport Failure Router Failure
Internet MPLS Internet MPLS Internet MPLS
Site Network Site Network Site Network
Redundancy – Extended Transports
• Each WAN Edge router is • SD-WAN tunnels are built through

connected to a given transports local and remote transports
Circuit Failure Transport Failure Router Failure
Internet MPLS Internet MPLS Internet MPLS
Site Network Site Network Site Network
Redundancy – Path and Headend
• WAN Edge routers leverage BFD for
detecting end-to-end tunnel liveliness
Data
Center
• If intermediate network path through
the SD-WAN fabric fails or if the
remote-end WAN Edge router (e.g.
data center) fails, BFD hellos will time
Internet MPLS out and remote site WAN Edge router
will bring down its relevant IPSec
tunnels
• Traffic will be rerouted after the failed
condition had been detected
Remote
- BFD timers can be tweaked for faster
Site detection
Redundancy – vSmart Control Controllers
vSmart • vSmart controllers exchange OMP
Controllers
messages and they have identical view
Control Plane
of the SD-WAN fabric
Data Plane
• No impact as long as WAN Edge
Cloud
routers can connect to at least one
Data Center vSmart Controller
• If all vSmart controllers fail or become
Data Center
unreachable, WAN Edge routers will
MPLS 4G
continue operating on a last known
INET
good state for a configurable amount of
Small Office time
Home Office - No changes allowed
Campus
Branch
Redundancy – vManage System
vManage • vManage servers form a cluster for
Cluster
redundancy and high availability
Management Plane
Data Plane • All servers in the cluster act as
active/active nodes
- All members of the cluster must be in
Cloud
Data Center
the same DC / metro area
• For geo-redundancy, vManage servers
operate in active/standby mode
MPLS 4G
Data Center - Not clustered
INET - Database replication between sites
Small Office
• Loss of all vManage servers has no
Home Office impact on fabric operation
Campus - No administrative changes
Branch - No statistics collection
vAnalytics Collection and Value
vAnalytics vManage
SD-WAN
Data Export Telemetry
Fabric
Visibility • Requires opt-in

• Cloud only
What-If
• Enterprise License tier
Recommendations
Forecasting
vAnalytics Main Characteristics
Network Centric Application/Flow Centric
• Site Availability • Based on DPI and cflowd
• Network Availability • Bandwidth Usage
- Top sources, destinations apps
• Site Usage Analysis
- Per-Site basis
- Top sites by bandwidth consumption
- Historical bandwidth consumption • Application Performance
- Application to tunnel binding and
• Carrier Performance
performance information
- Approute stats on a per-carrier
basis • Anomaly Detection
- Carriers health ranking - Baseline of application usage
- Anomaly detection based on overall
application usage (by application
family, by site)
Customer Deployment
Use Case: Retail
Retail Deployment - Details
SCALE
Controller Deployment
FinServ Multi Segment Overlay Topology
Back Office WiFi Point of Sales
Controllers are mostly in cloud

Voice Management Network Hub and Spoke Topology
Direct Internet Access Cloud Security Transport
LTE
Direct Internet Access Dual Homed Single Home

From Stores with LTE backup
TIME TO MARKET
Legacy Design
DC Region 1 DC Region 2
Internet and cloud access from the datacenter
MPLS
Internet
………………
Store 1 Store 2 Store 3 Store n
Hybrid Internet only with Internet Only and
redundancy no redundancy
Current Design For Datacenters
Internet MPLS Active Standby Internet MPLS
Internet MPLS Internet MPLS

Edge Router Edge Router
eBGP iBGP eBGP iBGP
Core Routers Core Routers

DCI eBGP
eBGP eBGP
Firewall Firewall Firewall Firewall
DC1 - Subnet DC2 - Subnet
Current Design For Remote Sites
VLANs
MPLS Internet
PCI
Voice
Guest Wireless
Corporate Wireless
Management
Internet Access – Guest Active Backup
Router Router
Internet Access – Employees
Vendor/Partner Connectivity VRRP running for all
VLANs
Switches and L2 FW at
each remote location on
the LAN side
Pain Points
Insufficient
Bandwidth
Complex Limited Application

Operations Awareness
Retail
High Applications
Cost Pain Points Downtime
Limited Fragmented
Scale Security
No Cloud Apps
Readiness
Overall Retail Solution
AWS: Avail. Zone 1
HQ DC
*Spoke-to-hub
Branch 1 Data Plane
SD-WAN Fabric
AWS: Avail. Zone 2
3rd Party
Branch 2 Cloud Security
Backoffice VPN 1
Internet
WAN
POS VPN 2
Guest WiFi VPN 3

DIA
GRE or IPsec
*HQ/DC/Main sites has default fully meshed data plane

All sites in control plane session with both AZ’s
Controller Deployment in AWS
Management Subnet / VPN 512 Management Subnet / VPN 512
vBond vSmart vManage vBond vSmart vManage

Backup
IGW IGW
WAN Subnet / VPN 0 WAN Subnet / VPN 0
vBond vSmart vManage vBond vSmart vManage
Internet
Private IP
Elastic IP
vManage/vSmart are configured with elastic IP of vBond to force
communication to pass though IGW (recording Private/Public)
Control Plane Sessions AWS: Frankfurt
HQ DC
Branch 1 Temporary vBond Connection

Permanent to both vSmarts
Permanent to vManage
SD-WAN Fabric
vBond + vSmart on every TLOC AWS: Dublin
vManage only on one TLOC / Edge
Branch 2
Backoffice VPN 1
Internet
WAN
POS VPN 2
Guest Wifi VPN 3

DIA
*HQ/DC/Main sites has default fully meshed data plane

TLOC – Color public-internet
All sites in control plane session with both AZ’s
Datacenter Migration
Internet MPLS
Default Pointed Default Pointed

to MPLS and Internet MPLS to MPLS and
Edge Router
Internet Routers Internet Routers
eBGP iBGP
eBGP
eBGP
eBGP
Firewall Firewall
DC subnets, summary DC subnets, summary

and default routes and default routes
advertised to WAN Edges advertised to WAN Edge
ACI Fabric
DC Subnets
Problem you have to fix…
Legacy design don’t have any segmentation while

with SD-WAN you are likely to introduce it
Option1: Extranet
Control Policy to leak routes
VPN100
between VPNs
Datacenter
Extranet
VPN1
VPN2 Routes
VPN100
Internet SD-WAN MPLS VPN3
Fabric
VPN4
Issue with Extranet

In future if you wanted to do
segmentation in DC, you vSmart
Remote
would have required a design
Sites
change
SD-WAN
VPN1 VPN3
OMP
VPN2 VPN4
LAN
Option2: 1 to 1 VPN Mapping
L2 Switch
Firewall
eBGP session per-VPN, allowing DC subnet VLANs mapped to a VPN and branch routers run
inbound separate VRRP for each application
VLANs
PCI
Voice
Guest Wireless
Corporate Wireless
Management
Datacenter Internet Access – Guest Branch
Internet Access – Employees
Vendor/Partner Connectivity
Control Policy used for Topology Creation
Data Plane or VPN Plane Topologies
Data Plane or Individual VPNs subject to specific topologies / connectivity models
• Fully meshed fabric data plane • Restricted fabric data plane

• Individual VPNs can use any • Individual VPNs restricted to
topology connectivity model used by
underlying fabric
Data Plane or VPN Plane Topologies
Data Plane or Individual VPNs subject to specific topologies / connectivity models
Site-Id: 100
Filter/Reassign Routes / Attributes

Filter/Reassign TLOCs / Attributes
Site-Id: 30
Site-Id: 10
Site-Id: 20
Data Plane and VPN Hub-and-Spoke Topologies
Policy Policy
lists control-policy restricted_data_plane
sequence 10
tloc-list hub-site_tlocs
match tloc
tloc 1.1.1.1 color red encap ipsec preference 100 site-list hub_sites
tloc 2.2.2.2 color red encap ipsec preference 100 !
tloc 3.3.3.3 color red encap ipsec action accept
! !
site-list branch_sites !
site-id 1000-2000 sequence 20
! match route
site-list hub_sites site-list branch_sites
!
site-id 1-100
action accept
! set
! tloc-list hub_site_tlocs
!
!
!
apply-policy sequence 30
site-list branch_sites match tloc
!
control-policy restricted_data_plane out
action reject
! !
! !
VPN 1 Full Mesh and VPN 2 Hub-and-Spoke Topologies
Loose Hub-and-Spoke Strict Hub-and-Spoke
Spokes communicate via hub(s) No spoke to spoke communication
Policy
Policy
lists
lists
vpn-list VPN2
vpn-list VPN2
vpn 2
vpn 2
!
!
site-list hub_sites
site-list branch_sites
site-id 1-2
site-id 100-200
!
!
!
!
control-policy vpn_multi-topology
control-policy vpn_multi-topology
sequence 10
sequence 10
match route
match route
site-list hub_sites
vpn-list VPN2
vpn-list VPN2
!
!
action accept
action accept
!
set
sequence 20
tloc 1.1.1.1 color red
match route
!
!
!
action reject
!
!
Brownout Mitigation
 WAN Routers continuously vManage App Aware Routing Policy
App A path must have:
perform path liveliness and
Latency < 150ms
quality measurements Loss < 2%
Jitter < 10ms
Internet
Remote Site
MPLS
Path 2 Data Center
4G LTE
Path3: 140ms, 1% loss, 10ms jitter SD-WAN IPSec Tunnel
Path Quality and Liveliness Detection
Multiplier (n)
• Each WAN Edge router sends BFD hello
packets for path quality and liveliness
detection
- Packets echoed back by remote site
Hello Interval (ms) • Hello interval and multiplier determine how
Liveliness
many BFD packets need to be lost to
Quality declare IPSec tunnel down
App-Route Multiplier (n)
• Number of hello intervals that fit inside poll
interval determines the number of BFD
Poll Interval Poll Interval Poll Interval (ms)
packets considered for establishing poll
interval average path quality
• App-route multiplier determines number of
poll intervals for establishing overall average
Hello Interval (ms) path quality
Application Aware Policy Example
policy
sla-class BE
loss 8 Default Timers
latency 200 bfd app-route multiplier 6
! bfd app-route poll-interval 600000 (10
sla-class P1 minute window)
loss 3
latency 100 SLA will be measured with a running
! average of a 60 minutes sliding
window
app-route-policy REI-branch-aar
SLA depends on…

vpn-list REI
sequence 10
match
dscp 46
Configured Timers
!
action bfd app-route multiplier 2
sla-class P1 bfd app-route poll-interval 120000 (2
! minute window)
!
sequence 100 SLA will be measure with a running
action average of a 4 minute sliding window
sla-class BE
!
SD-WAN Internet Breakout Options
Local Breakout using a Default Route
• Static route in Service VPN

- Can be default or more granular
Internet
• Redirects traffic to interfaces in VPN 0
Branch - Interfaces must have NAT enabled
vpn 0 - Multiple interfaces enables per-flow load-
interface ge0/0
nat
sharing
! - Relies on VPN 0 routing table
vpn 1
ip route 0.0.0.0/0 vpn 0
• Can be complemented with a Tracker
to monitor Internet availability beyond
first hop gateway
Local Breakout using Data Policy
• Policy now redirects instead of static
Color public-internet route
Internet
Color blue
- In case local exit fails, lookup can fall back
to local service VPN routing table
Branch
• Redirects traffic to interfaces in VPN 0
WAN Edge
vpn 0 - Interfaces must have NAT enabled
interface ge0/0
nat - Multiple interfaces enables per-flow load-
sharing
vSmart
policy - Relies on VPN 0 routing table
data-policy internet-breakout
vpn-list VPN1 • Can be complemented with a Tracker
to monitor Internet availability beyond
sequence 10
match source-ip 10.0.0.0/8
!
action accept
first hop gateway
• Local TLOC to be used can be
nat use-vpn 0
local—tloc public-internet
specified
Using a Tracker to ensure functional Internet Access
Ge0/0
Internet
Ge0/1
Branch • BFD only manages TLOC reachability

WAN Edge • Different mechanism needed to qualify DIA
System
tracker google
connection as functioning
endpoint-dns-name www.google.com
interval 60 (default, seconds) • Tracker uses native DIA path for probes
multiplier 3 (default)
threshold 300 (default, ms) - Configured on a per Interface basis
!
! - Uses HTTP Probes only
vpn 0
interface ge0/0 - Relies on VPN 0 routing table
nat
tracker google • With Tracker down, all routes resolving
onto a tracked interface are invalidated
vSmart
Localizing the WiFi Local Breakout / DIA
Policy
lists
Branch vpn-list VPN3
vpn 3
VPN 1 !
WAN
Backoffice POS VPN 2
Internet site-id 100-200
VPN 3 !
DIA !
control-policy localize_wifi
sequence 10
match route
vpn-list VPN3
!
action reject
!
Guest Wifi !
!
!
apply-policy
site-list branch-sites
control-policy localize_wifi in
Cloud Security: Standard Routing with HA
Branch 3rd Party
Cloud Security
VPN 1
WAN
Internet
VPN 3
DIA
GRE or IPsec
vpn 0
interface gre1
ip address 10.0.0.1/24
Guest Wifi keepalive 10 60
tunnel-source ge0/0
tunnel-destination 2.1.1.1
no shutdown
!
interface gre2
…
!
!
vpn 1
ip gre-route 0.0.0.0/0 vpn 0 interface gre1 gre2
Cloud Security: Policy-Driven with HA
Branch 3rd Party
Cloud Security
VPN 1
WAN
Internet
VPN 3
DIA
GRE or IPsec
WAN Edge vSmart

vpn 1 policy
service FW interface gre1 gre2 data-policy Cloud_Security
vpn 0 vpn-list vpn_3
Guest Wifi
interface gre1 sequence 10
ip address 10.0.0.1/24 match source-ip 10.0.0.0/8
tunnel-source-interface ge0/0 !
tunnel-destination 2.1.1.1 action accept
no shutdown set
! service FW local
interface gre2 !
… !
! !
! default-action accept
High Availability
SD-WAN • VRRP between WAN Edge routers

- Operates per-VLAN
Fabric
• VRRP Active WAN Edge router responds to
ARP requests for the virtual IP and virtual
WAN WAN MAC*
Edge A Edge B
VRRP • Prior to 18.3.0, in case of failover, new
VRRP Active WAN Edge (vEdge) router
sends out gratuitous ARP to update ARP
table on the hosts and mac address table
Host
on the intermediate L2 switches
* Virtual MAC requires minimum 18.3.0 code on vEdge
Transport Redundancy - Meshed
• WAN Edge routers are connected to all
the transports
• When transport goes down, WAN Edge
MPLS INET routers detect the condition and bring
down the tunnels built across the failed
transport
WAN WAN - BFD times out across tunnels
Edge Edge
• Both WAN Edge routers still draw the
traffic for the prefixes available through
the SD-WAN fabric
DC LAN Network • If one of the WAN Edge routers fails,
second WAN Edge router takes over
forwarding the traffic in and out of site
- Both transport are still available
Transport Redundancy – TLOC Extension
• WAN Edge routers are connected only
to their respective transports
• WAN Edge routers build IPSec tunnels
MPLS INET across directly connected transport
and across the transport connected to
the neighboring WAN Edge router
WAN WAN
- Neighboring WAN Edge router acts as an
Edge Edge
underlay router for tunnels initiated from
the other WAN Edge
• If one of the WAN Edge routers fails,
second WAN Edge router takes over
Site Network
forwarding the traffic in and out of site
- Only transport connected to the remaining
WAN Edge router can be used
TLOC Extension Configuration
vpn 0 vpn 0
interface ge0/0 interface ge0/0
description MPLS tunnel description INET tunnel
ip address 100.65.51.1/30 Add route to reach ip dhcp-client
Do not forget NAT
tunnel-interface WAN Edge2 mpls nat
encapsulation ipsec tunnel end-point !
color mpls restrict tunnel-interface
[service list] MPLS INET encapsulation ipsec
! color biz-internet
interface ge0/2 [service list]
description INET tunnel !
ip address 10.5.51.51/24 interface ge0/2
! ge0/0 ge0/0 ip address 10.5.51.52/24
tunnel-interface 100.65.51.1/24 dhcp tloc-extension ge0/0
encapsulation ipsec ge0/2 ge0/2
no shutdown
color biz-internet 10.5.51.51/24 10.5.51.52/24 !
[service list] interface ge0/3
! description MPLS tunnel
interface ge0/3 ip address 10.5.52.52/24
ip address 10.5.52.51/24 ge0/3 ge0/3 tunnel-interface
10.5.52.51/24 10.5.52.52/24
tloc-extension ge0/0 encapsulation ipsec
no shutdown WAN Edge1 WAN Edge2 color mpls restrict
! [service list]
ip route 0.0.0.0/0 100.65.51.2 Extended MPLS TLOC /24 subnet needs to be !
ip route 0.0.0.0/0 10.5.51.52 advertised in MPLS core network ip route 0.0.0.0/0 10.5.52.51
High Availability with DPI and Zone Based Firewall
For DPI and ZBF, traffic has to Data Center

be symmetric
MPLS INET
Inbound use higher preference
on WAN Edge A to attract traffic
Outbound WAN Edge A is the
VRRP Active Router
vpn 0
interface interface-name
tunnel-interface
encapsulation (gre | ipsec)
preference number
WAN Edge A WAN Edge B weight number
VRRP Active VRRP Standby
Preference vs Weight
Preference
• TLOCs with the highest preference are chosen to forward outbound traffic
• If all TLOCs have the same preference traffic flows are evenly distributed
among the tunnels, using ECMP.
• Configured under the tunnel interface
Weight
• Weight is used to achieve unequal cost multipath
• Flows are distributed across TLOCs based on the weight ratio
• For example, if TLOC A has weight 10, and TLOC B has weight 1, and
both TLOCs have the same preference value, then roughly 10 flows are
sent out TLOC A for every 1 flow sent out TLOC B.
Overall Retail Solution AWS: Frankfurt
Controllers: AWS Hosted
HQ DC
*Spoke-to-hub
Data Plane
Branch 1
Control Policy: Hub-and-Spoke Topology
Data Policy:
SD-WANDirect
Fabric Internet Access
Data Policy: Wifi / Cloud-Security Breakout AWS: Dublin
Branch 2 3rd Party
Cloud Security
Backoffice VPN 1
Internet
WAN
POS VPN 2
Guest Wifi VPN 3

DIA
GRE or IPsec
WAN Edge: Segmentation, DIA, Tunnel to Cloud Security
Wrap up
Key Messages
Cisco SD-WAN Solution helps you to:
Reduce Cost
Operate Faster with Security
Integrate Latest Cloud and Network Technologies
Your SD-WAN Learning Map at CLEUR
Monday Tuesday Wednesday Thursday Friday

TECCRS-2014 BRKRST-2560
Deep Dive BRKRST-2559
Analytics / ML
On-Prem
Deployment
BRKCRS-2112
Serviceability BRKCRS-2117
Design
TECCRS-2191 BRKCRS-2114 Deployment
Deployment / BCP BRKCRS-2111 Security
Migration
TECSEC-2355
Security BRKRST-2558 BRKCRS-2113
BRKCRS-2110 SD-WAN as a Cloud onRamp
The Foundation Managed Service
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
cs.co/ciscolivebot#TECCRS-2014
Complete your online
session survey
• Please complete your Online Session
Survey after each session
• Complete 4 Session Surveys & the Overall
Conference Survey (available from
Thursday) to receive your Cisco Live T-
shirt
• All surveys can be completed via the Cisco
Events Mobile App or the Communication
Stations
Don’t forget: Cisco Live sessions will be available for viewing

on demand after the event at ciscolive.cisco.com
Continue Your Education
Demos in Meet the Related

Walk-in
the Cisco engineer sessions
self-paced
Showcase labs 1:1
meetings
Thank you

Teccrs 2014

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Teccrs 2014

Uploaded by

Copyright:

Available Formats

TECCRS-2014

Hardware Centric Software Driven

Network Intent Business Intent

Devices & Things

More user, things and applications, everywhere

DEVICES DNA Center

Policy Automation Analytics

Transport Independent Superior Security aAchitecture:

• Simple: Single Hop, Input / Output

Critical Applications SLA Bandwidth Augmentation Secure Segmentation

SDWAN Tunnel SDWAN Fabric

Regional Secure Perimeter Guest WiFi DIA & DCA

Over The Top Business VPN Extension over Last Mile

Internet MSP MPLS

2. Cisco Umbrella Cisco

vEdge vEdge Cisco Router vEdge ISR4K SD-WAN

Thin Edge with vEdge, Rich End-to-end experience with full

Platform: Platform: Management:

3rd Party • Orchestrates control and

3rd Party • Facilitates fabric discovery

vManage Cisco WAN Edge

3rd Party • Single pane of glass for Day0,

Steps in Establishing Cisco SD-WAN Fabric

Instantiate Control Establish Control Establish Data

Deploy Deploy Deploy

vManage vManage vManage

vSmart vBond vSmart vBond vSmart vBond

On-Premise/SP Hosted Cloud Hosted

ESXi or KVM AWS or Azure

Physical Server Container Container

Configure Device Template

• Virtual machine or container

• Automated certificate signing through

vBond vSmart • Controllers list is distributed by vManage

• Device Certificate* – Own identity (SHA256)

In Software Provided by vManage CA * Can use Enterprise CA Certificate

• Device Certificate – Own identity (SHA1)

Device • DigiCert* Root Chain – Trust for controllers’

* Can use Enterprise CA Root Chain. Can be loaded

• Device Certificate – Own identity (SHA256)

Device • DigiCert* Root Chain – Trust for controllers’

* Can use Enterprise CA Root Chain. Can be loaded

• Device Certificate – Own identity (SHA256)

* Can use Enterprise CA Root Chain. Can be loaded

• Device Certificate – Own identity (SHA256)

* Can use Enterprise CA Root Chain. Can be loaded

vManage • Administrator defined controllers

• Signed WAN Edge list (white-list)

vManage  vSmart validates:

 WAN Edge validates:

• Secure Channel to SD-WAN DTLS only

WAN Edge WAN Edge

vBond vSmart vManage

Red signifies primary protocol or first port used

• Administrator decides on identity trust

• WAN Edge list and identity trust are

Configure Device Template

DMZ (NAT 1:1) 5

The router contacts a DHCP

• DHCP is not enabled on CE to PE

• Config file (which includes basic

WAN Edge • Router has all required information

A / 2001 <-> Z / 3001 / A / 2001 <-> Z / 3001 B / 90