Professional Documents
Culture Documents
BRKRST 2559
BRKRST 2559
SD-WAN On-Prem
BRKRST-2559
Cisco Webex Teams
Questions?
Use Cisco Webex Teams to chat
with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Agenda
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Architecture
Orchestrator
Management Plane
vManage • Orchestrates control and
• Single pane of glass vBond management plane
• Centralized provisioning
• First point of authentication
• Policies and Templates
• Facilitates NAT traversal
Control Plane
• Facilitates fabric discovery
• Disseminates control plane
Zero Touch Provisioning
information vSmart Controllers • Facilitates device onboarding
• Implements and distributes policies ZTP
MPLS 4G Data Plane
INET • Diversity of Physical or
Virtual appliances
WAN Edge
• Builds IPsec tunnels and
exchanges user traffic
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Controllers Deployment Options
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Colors, Address
Assignments, and
Connectivity
On-Prem Design Consideration
• How to connect WAN Edge devices to controllers?
• Internet
• MPLS
• Multiple Transports
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Transport Colors and Control Connections
Local Color: Public Local Color: Private Local Color: Private
Controller Color: Public Controller Color: Public Controller Color: Private
Use: Public IP Use: Public IP Use: Private IP
INET MPLS
MPLS
1:1 1:1 1:1 1:1
NAT NAT NAT NAT
Option Controller’s IPs Behind NAT Color Type Reachable from Reachable from
INET MPLS
A Public No Public Yes Only if advertised
• Prefer designs with control connections over multiple transports for better resiliency
• Option A) is the cleanest/simplest
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Controllers Placement in On-Prem Environment
HQ • Ensure proper connectivity
Internet to controllers from private
Firewall transport
vBond vManage vSmart
control
connection
INTERNET
DMZ
WAN Firewall
Edges
WAN CE Branch
Core
MPLS
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Using Loopback for TLOC Termination
• Problem: TLOC configuration on WAN interface locks down the interface – control
connections are not routed through.
IF Service Transport
(VPNs) (VPN0)
X
IF TLOC IF MPLS
OOB Mgmt
(VPN512)
IF
control
connection
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Using Loopback for TLOC Termination
• Problem: TLOC configuration on WAN interface locks down the interface – control
connections are not passed through
• Solution: Configure TLOC interface on loopback
TLOC
IF
Loopback
IF Transport IF MPLS
(VPN0)
OOB Mgmt
(VPN512)
IF
control
connection
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Connecting Controllers Without WAN CE Router
HQ vBond vManage vSmart
INTERNET
DMZ
WAN Edges Firewall
Loopback
Core Branch
MPLS
Firewall
WAN Edge
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Deployment
Requirements
Controllers’ Requirements
vNIC 2 vNIC 1
• SSD-based storage required for all controllers
VPN0 VPN512
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Verifying vManage System Requirements
• Private lab setup for learning purposes will work with less resources.
• * vManage Cluster requires dedicated interface for message bus.
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Verifying vBond System Requirements
1-50 2 4 GB 10 GB 1 Mbps 2
51-250 2 4 GB 10 GB 2 Mbps 2
251-1000 2 4 GB 10 GB 5 Mbps 2
1001+ 4 8 GB 10 GB 10 Mbps 2
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Verifying vSmart System Requirements
1-50 2 4 GB 16 GB 2 Mbps 2
51-250 4 6 GB 16 GB 5 Mbps 2
251-1000 4 16 GB 16 GB 7 Mbps 2
1001+ 8 16 GB 16 GB 10 Mbps 2
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Performing Controller Installation
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
Managing Smart Account
& Virtual Accounts
Cisco Smart and Virtual Accounts
PnP Portal
Obtain
SD-WAN
Software
Smart Account
Management
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Workflow Overview
Define Controller Profile
1 (vBond, org-name, root CA)
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Defining Controller Profile
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Controller Profile Details
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Adding Brownfield Devices to PnP Portal
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Adding Brownfield Devices to PnP Portal (Cont.)
• On IOS-XE platforms
running 16.6.1 or more use:
show crypto pki certificates
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
Obtaining License / Provisioning File
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Certificate
Authorities
Certificate Authority Options
• Cisco PKI can be used for on-prem controllers deployment.
• CSRs can be automatically signed using configured Smart
account and internet connectivity from vManage.
• Manual signing is supported via PnP portal.
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Utilizing Cisco PKI
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Utilizing Cisco PKI – Manual Method
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Manually Submiting CSR to Cisco PKI
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Utilizing Cisco PKI - Downloading Signed Cert
• When approaching expiration date, make sure new CSRs are generate and new
certificates obtained and installed.
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Using Enterprise CA
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Utilizing Enterprise CA
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Zero Touch
Provisioning
Zero Touch Provisioning – vEdge HW Appliance
Public or On-Prem Controllers
ZTP vBond
• Public ZTP vBond can redirect
to cloud hosted or On-Prem
controllers.
1 2
3 4 5
Full Registration
and Configuration • Additional devices can be
vEdge Router
associated with the customer
using the PnP Connect portal
Option1: Option2:
DHCP on WAN interface Discover local addressing via ARP
DNS to resolve ztp.viptela.com Google DNS: resolve ztp.viptela.com • ZTP for vEdges can be
deployed also On-Prem
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Configuring On-Prem ZTP vBond Server
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Obtaining Signed Certificate by Trusted CA
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Obtaining Signed Certificate by Trusted CA (Cont.)
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Uploading The ZTP Whitelist Chassis File
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Zero Touch Provisioning – WAN Edge Appliance
Controllers • The PnP Connection Manager
can redirect to cloud-hosted
Connection
Manager
or On-Prem controllers.
1 2
3 4 5
• New devices are linked to
organization using the Smart
Account when placing order.
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
ZTP – Bootstraping With Configuration File
<… output omitted …>
#cloud-boothook
system • Upon bootup, the router
personality
device-model
vedge
vedge-ISR-4321
searches bootflash: or usbflash:
host-name
system-ip
WanEdge
10.255.255.121
for filename ciscosdwan.cfg.
site-id 21
organization-name "CLEUR 2020 BRKRST - 2559"
console-baud-rate 9600
vbond 203.0.113.3 port 12346
! • The config file with interface
!
interface GigabitEthernet0/0/0 configuration, Root CA,
no shutdown
ip address 198.0.51.10 255.255.255.0 Organization Name, vBond
exit
! information, is fed into the PnP
ip route 0.0.0.0 0.0.0.0 198.0.51.1
<… output omitted …> process.
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Designing for High
Availability and
Scale
Controllers High Availability Overview
Manual DB
vSmart
replication
Active Active DB Replication
Cold Standby
Standby Cluster
Active
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
vBond High Availability and Scale
• Default behavior: WAN Edge tries to resolve and connect to all known vBond IPs on
all WAN interfaces. Connection is transient.
• Scale approach:
• Configure regional domain name to point to specific regional vBond pair
• Rely on DNS A records or define manual host entry
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
vSmart High Availability and Scale
vSmart
• Default behavior:
• WAN Edge connects up to two vSmarts on each transport
• Example: WAN Edge with two transports == 2 control connections and 1 OMP session per vSmart
vSmart vSmart
• No control over vSmart preference
DTLS/TLS
WAN Edge
OMP + DTLS
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Controller Groups
AMER EMEA
vManage vBond Group2
Group1 vSmart
vSmart APAC
vBond vBond
Group3
Group 2,1
vSmart
FQDN AMER Group 1,2 FQDN EMEA
WAN Edge
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Standalone vManage – Disaster Recovery
• Prerequisites: vManage
• Same SW version Active
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
Managing Backup of Active vManage Controller
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Activating Standby vManage Controller
• Import backup to standby vManage
• Check all services are running using #request nms all status
• Under Configuration>Certificates>Controllers edit existing vBond entries by
retyping mgmt IPs and credentials.
• Bring up vManage tunnel-interface
• Send the updated device list to vBond controllers
• Under Configuration>Certificates>WAN Edge List
select Send to Controllers
• Invalidate failed vManage controller
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Taking VM Level Snapshots
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Designing
vManage Cluster
with High Availability
vManage Cluster
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Understanding the vManage Cluster
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
vManage Cluster Design – Basic Deployment
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
vManage Cluster Design – Increasing Stats DB
Performance and Scale
• When improved performance and scale of Statistics DB is required
• Configuration DB redundancy is not provided
• Failure of first node will prevent management until recovery
• In case of other node failure, cluster can support up to 6000 devices
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
vManage Cluster Design – Large Deployment
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
vManage Cluster
Disaster Recovery
vManage Cluster Disaster Recovery
• Problem: Cluster nodes must be part of same DC due to low latency requirements.
Single cluster does not fulfill DR requirements. Need for automatic failover.
• Solution: Primary cluster, standby cluster, and arbitrator instance, which performs
automatic failover in case of failure
Arbitrator
DB Replication
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
Understanding DR Arbitrator
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
Primary vManage Cluster Active
Arbitrator
DC1 DC2
vBond vSmart vBond vSmart
DC1 DC2
vBond vSmart vBond vSmart
Active Cluster
Arbitrator
Switch over
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Scenario 2) Failed Connectivity With Arbitrator
Active Cluster
Arbitrator
Switch over
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Scenario 3) Failed Connectivity With Arbitrator
or Failed Arbitrator
Active Cluster
Arbitrator
Switch over
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Next Steps
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
BRKRST-2791
Building and using Policies with Cisco SD-
BRKRST-2377 WAN
08:00
SD-WAN Security 08:00 BRKRST-2560
Keynote 09:30
SD-Wan Machine Analytics, Machine
08:00
Learnings and IA
BRKCRS-1579 BRKRST-2096
SD-Wan Proof Of Concept
11:00
SD-WAN Powered by 11:00 BRKRST-2095 BRKRST-2093
Meraki SD-WAN Routing 16:00 Deploy, monitor and troubleshoot
11:00 BRKRST-2091
BRKRST-2041 Migration
BRKARC-2012 SD-WAN Datacenter and Branch 09:00
WAN Architecture 11:00 ENFV Architecture, Configuration and
11:00 Integration Design
troubleshooting
and Design Principal
BRKRST-2559
BRKCRS-2110 3 Steps to design SD-WAN On Prem
14:00
Delivering Cisco Next 14:00 BRKRST-3404 BRKRST-2097 BRKOPS-2826
gen SD-WAN with How to choose the 16:00 Conquer the Cloud with SD-WAN SD-WAN as Managed Services 11:00
14:45
Viptela correct branch device BRKRST-2095
SD-WAN Routing Migrations
16:45
BRKCRS-2113 Keynote 17:00
Cloud Ready WAN for 17:00 Cisco Live
IAAS and SAASA with Celebration
Cisco SD-WAN 18:30
SD-WAN
#CLEMEA
Breakouts
Complete your
online session
survey • Please complete your session survey
after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live t-shirt.
• All surveys can be taken in the Cisco Events
Mobile App or by logging in to the Content
Catalog on ciscolive.com/emea.
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
Continue your education
Demos in the
Walk-In Labs
Cisco Showcase
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Thank you
Loopback Interface - Bind mode
VPN 0
LAN Core
MPLS1
L0 T1
VPN1
MPLS2
L1 T2
VPN2
Internet
L2 T3
LAN Core
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
Loopback Interface – Unbind Mode
• In case of unbind mode, loopback interface is not bound to any physical interface
• Traffic destinated to loopback can go through any physical interface (Based on hash
lookup)
• This can be used when there are multiple transports available to same provider
VPN 0
LAN Core
T1
VPN1
MPLS
L0 T2 Provider
VPN2
T3
LAN Core
BRKRST-2559 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 89