SD-WAN Booklet PDF

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 173

For details contact: www.networkershome.com | info@networkershome.

com | Mob: +91 9611027980 | +91 9354284954


For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
W X P

Enterprise Applications

Remote Sites Data Center

WAN

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
W X P

Internet Cloud Enterprise

DC2
DC1
Home Users
Remote Sites

SD-WAN

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Computer

Individual
Components as a System

Users configure individual


components (network devices)
in order to use the system (the
network).

System

Individual
Components

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Users simply signify intent to use the
system (the network) and it
configures all underlying
components (network devices).

System

Individual
Components

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Nowadays
In the old days - Drivers are aware of road blocks and construction works
- Drivers only know that different roads exist
- Traffic Jams and road accidents
- Real-time auxiliary information is not available
- Toll Taxes and much more info

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco Viptela VS Cisco Meraki
Enterprise-level SD-WAN solution SD-WAN solution with a basic level of
supporting complex WAN topologies customization designed for small and
with a high degree of customization medium sized organizations

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vManage

On-prem Cloud Multi-tenant

ANY DEPLOYMENT

Routing Analytics Cloud (IaaS) Segmentation Security

ANY SERVICE

Internet MPLS 4G/5G Satellite

ANY TRANSPORT

Campus DataCenter Industrial SOHO Cloud

ANY LOCATION

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
GUI Automation
Management Plane

vAnalytics vManage
Orchestration Plane
vBond

Control Plane
vSmart
Controllers

MPLS Internet 4G/5G

vEdge Routers
Data Plane

Cloud DC Campus Branch

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco vSmart
OM
P
Up

te
da

da Control Plane
te

Up
P
OM

Overlay Tunnel 1
Routing Routing
Information Overlay Tunnel 2 Information

Data Plane

Campus Network Branch Network

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco vSmart
Controller

Each vEdge keeps one


permanent connection
to the vSmart controller MPLS Internet
via each available
transport

vEdge 1 vEdge 2

Campus Network

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Order of Deployment

Deploy Controllers Deploy Large Sites Deploy Branches


vManage vEdge vEdge

SOHO

Datacenter HQ Campus Branch

vBond vSmart Cloud

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
On-prem Deployment Cloud Hosted
vSmart vSmart
Controllers Controllers
vManage vBond vManage vBond

VM VM VM VM VM VM

ESXi or KVM Azure or AWS

Physical Server

vBond

UDP UDP UDP UDP UDP UDP


12346 12346 12346 12346 12346 12346

UDP UDP UDP UDP UDP UDP


12346 12446 12546 12646 12346 12446

UDP UDP
if DTLS
12346 12346
or
TCP TCP
if TLS
Random 23456
vManage vSmart
(4 cores) (2 cores)

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
SD-WAN Controllers SD-WAN Controllers
Method
2

Data Center Public Cloud Public Cloud


Public IPs Public IPs

MPLS Internet MPLS Internet

vEdge vEdge
Method
1
Remote Site Remote Site

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vEdge Onboarding Options

Automated Deployment Manual Deployment

Zero-Touch
Plug-and-Play
Provisioning Bootstrap Manual
(PnP)
(ZTP)

Cisco XE Devices Viptela Devices Cisco XE Devices Viptela Devices


Cisco XE Devices

PnP/ZTP service vBond vManage vSmart

PnP

Zero-Touch Autentication Netconf OMP Peering


Provisioning 1 transient 1 permanent 1 permanent
1 transient connection connection connection per
connection transport

2
4
1

vEdge

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Any controllers deployment
- Public Cloud
- Private Cloud vManage vBond vSmart
- On-premises
vEdge vEdge Control Plane

vEdge

Any overlay topology


- Hub and Spoke
vEdge vEdge
- Full mesh
- Partial Mesh

TLOCs TLOCs
Any underlay transport T1 T1

- Broadband T2
INET
SOHO
- MPLS HQ
- 4G/5G T1
TLOCs T1
T2
MPLS TLOCs
T2

DC Campus

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Quality of
Service
Remote (QoS) Data
Site Center

Application Application Circuits


Visibility Experience Quality

Clients Servers

Network
SLA

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
WAN Edge Router

Service Provider
QoS classes

SP4
Copy original DSCP marking
SP3
into the outer DSCP
Copy SP2

SP1
DSCP
DSCP

DSCP

IP Packet IP Packet
Ingress
Original Packet
Egress MPLS
Interface Encapsulated Packet Interface

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Remote Site Regional Hub
SD-WAN
tunnel

Transport
1
SD-WAN fabric detects the max path MTU
2
Apps send large packets

3
MTU exceeded, fragmentation required

4
Apps reduce packet size

5
Applications traffic send with correct MTU
No fragmentation required
Clients Servers

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For every block of 4 packets, One lost packet out of the
one parity packet is inserted four can be reconstructed

... 2 1 P 4 3 2 1 4 3 P 1

P 4 3 2 1
IPsec tunnel

Overlay fabric
vEdge vEdge
Sender Receiver
Site Site

Packets are sent over both overlay Duplicated packets are dropped
tunnels at the sending vEdge at the receiving vEdge

4 3 2 1 4 3 2 1

4 3 2 1
IPsec tunnel 1
IPsec tunnel 2

vEdge 4 3 2 1 vEdge
Sender Receiver
Site Site

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
App X must have:
vBond vSmart vManage
Latency <= 200ms
Packet Loss <= 3%
Jitter <= 15ms

Control Plane Real-time


path measurements
Path 1: Jitter 25ms
Path 2: Jitter 18ms
Path 3: Jitter 12ms

1
th
Pa
Broadband

Path 2

4G/5G
Branch Regional Hub
Pa
th
3

MPLS

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
TCP Optimized TCP TCP
connection connection connection

IPsec tunnel

Overlay fabric
vEdge vEdge
Users Servers

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vmware
aws

Salesforce Google Cloud

Software Infrastructure
Office 365 as a Service as a Service Azure
SaaS IaaS

Reliable and Secure


connection

Branch / Campus / Data Center

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Azure aws
vmware Google Cloud Salesforce

Cisco
SD-WAN

Campus Data Center Remote Site Private Cloud

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Microsoft Teams

Salesforce
Software-as-a-Service (SaaS)

Security Stack
DNS/Web layer
security

IPS/IDS

SD-WAN URL Filtering


Overlay
Firewalls
INTERNET

MPLS
vEdges vEdges
Branch
Datacenter

VPN 10 VPN 20
Employees Infra

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Microsoft Teams

Salesforce
Software-as-a-Service (SaaS)
Security Stack
DNS/Web layer
security

IPS/IDS

URL Filtering
SD-WAN
Firewalls Overlay
INTERNET

MPLS
vEdges vEdges
Branch
Datacenter

VPN 10 VPN 20
Employees Infra

Cloud
Security Provider

Cisco SD-WAN
DIA IPsec
tunnels

Internet Office 365

Branch

VPN 10
Employees

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
User defined 3rd party Custom 3rd party
automation controllers NMS tools OSS, BSS

RESTful APIs
vManage
NMS

vSmart
controllers

4G/LTE MPLS INET


Data Plane

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Phyton Phyton RESTful
SDK Script JSON APIs

API vManage

vSmart vSmart
controller controller

Engineer
SD-WAN Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
As far as there is IP reachability
btw T1 and T2 we represent this
as a logical link - tunnel
IP IP IP T1 T2 IP T1 T2 IP T1 T2 IP IP IP IP

T1 T2
vEdge-1 vEdge-2

Packets transverse the network


btw T1 and T2 with source IP T1
and destination IP T2.

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
T2 T3

T1 T4

Overlay

vEdge-2 vEdge-3
T2 T3

vEdge-1 vEdge-4

T1
Underlay T4

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Orchestration/Management

ts
en

Ro

O
g

em

M
ut
in

P
er

e
IPsec tunnels

tis

pe
pe

Ad
r

er
ve

ve
P

in
M

Ad

r tis

g
O

te

em
u
Ro

en
Overlay Network

ts
T1 T1
VPN0 INET VPN0
(Transport) (Transport)
T2 MPLS T2

Underlay Network

VPN1 VPN2 VPN1 VPN2

Local Networks Local Networks

VPN 10 Hub-and-Spoke

VPN 20 Custom

VPN 30 Custom

VPN 40 Custom

vEdge vEdge vEdge

vEdge vEdge

INET MPLS
VPN 0 - Transport

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
IPsec tunnel vSmart Routing table
1.1.1.0/24 via T1
DTLS tunnel 1.1.1.0/24 via T2
2.2.2.0/24 via T3
OMP peering 2.2.2.0/24 via T4

BFD session

OMP Update OMP Update


1.1.1.0/24 via T1 2.2.2.0/24 via T3
1.1.1.0/24 via T2 2.2.2.0/24 via T4

IPsec tunnel
with BFD

INET
T1 T3

T2 T4

VPN1 MPLS VPN1

IPsec tunnel
1.1.1.0/24 with BFD 2.2.2.0/24

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco vSmart
Controller

Each vEdge keeps one


permanent connection
to the vSmart controller MPLS Internet
via each available
transport

vEdge 1 vEdge 2

Campus Network

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart vEdge
controller router
OMP
DTLS/TLS
NETCONF
session
SNMP

vSmart
controller
OMP peering between vSmarts
OMP peering between vEdges

vSmart vSmart
controller controller

vEdge vEdge
router router

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
The vSmart controllers modify, store
and re-advertise the route information
received via OMP toward all other
vEdges
vSmart
controller

OMP peering
over DTLS

OMP Update
INET MPLS

TLOC T1 T2 TLOC

vEdge
router
FW Service
Connected Local-Networks (Connected,
Static, OSPF, BGP, etc)
Static Transport Locators (TLOCs)
Services (FW, IDS, IPS)
Dynamic Routing
(OSPF or BGP)

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vEdge-1 Transport
Locators
T1 (TLOCs)

T2
vEdge-3
T5

vEdge-2 T6
T3

T4 Tunnel Tunnel
source destination
Site-id 100 Site-id 200
IPsec

IPsec

IPsec

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
The vSmart controller
performs the OMP Best-Path
Algorithm
1.2.3.0/24 via T1
1.2.3.0/24 via T2
1.2.3.0/24 via T3
1.2.3.0/24 via T4
1.2.3.0/24 via T5
1.2.3.0/24 via T6
vSmart

Each vEdge advertises to vSmart that


By default, only the best 4
the subnet 1.2.3.0/24 is reachable via
routes (according to the OMP
all its TLOCs.
Best-Path Algorithm) are
advertised out.
T1 T2 T3 T4 T5 T6

vEdges

vEdges

1.2.3.0/24

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Route State
Prefer ACTIVE routes over STALE routes OMP
Best-Path
Route Resolvability
Next-hop TLOC must be reachable Selection
vSmart/vEdge v18.4
Source Preference and above
Prefer locally-sourced routes over vSmart-sourced

Admin Distance (AD)


Prefer OMP routes with lowest AD

Route Preference
Prefer OMP routes with highest route preference

TLOC Preference
By default 4 paths are Prefer OMP routes with highest TLOC preference
advertised by vSmart
send-path-limit [1-16]
Origin
Backup routes can also be Prefer OMP routes with best origin
advertised to vEdges for faster
convergence
send-backup-paths Tiebreaker
Prefer OMP routes with lowest origin System-IP
Origin (Connected, Static,
eBGP, OSPF Intra, OSPF Inter,
OSPF External, iBGP, Unknown) Tiebreaker
-> by Admin Distance
Prefer routes with lowest TLOC private address
-> then by Cost/Metric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Routing Table

vSmart

Inbound Centralized Outbound Centralized


Policy Policy
Modifies the routing information Modifies the routing information
before in enters the routing after the best-path selection
table of the controller has taken place

vEdge vEdge

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
TLOC {1.1.1.1, green, ipsec} vSmart
Private IP: 10.1.2.3 9.9.9.9
Private Port: 12346 Site-id 30
Public IP: 31.1.2.3
Public Port: 12346
Preference: 0
Site-id: 10
Tag: not set
Weight: 1
TLOC Route OMP
Update

NAT
10.1.2.3 31.1.2.3
150.2.2.2

Internet
Ge0/0 Ge0/1

vEdge-1 1.1.1.1 vEdge-2


1.1.1.1 T1 green 2.2.2.2
Site-id 10 ipsec Site-id 20

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
TLOC {2.2.2.2, green, ipsec}
Private IP: 150.2.2.2 vSmart
Private Port: 12346
Public IP: 150.2.2.2
9.9.9.9
Public Port: 12346 Site-id 30
Preference: 0
Site-id: 20
Tag: not set TLOC Route
Weight: 1

OMP
Update
NAT
10.1.2.3 31.1.2.3
150.2.2.2

Internet
Ge0/0 Ge0/1

vEdge-1 2.2.2.2 vEdge-2


1.1.1.1 green T2 2.2.2.2
Site-id 10 ipsec Site-id 20

NAT
31.1.2.3
10.1.2.3 150.2.2.2

Internet
Ge0/0 Ge0/1

vEdge-1 vEdge-2
1.1.1.1 T1 T2 2.2.2.2
Site-id 10 IPsec tunnel and a BFD Session Site-id 20

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
9.9.9.9
TLOC routes Site-id 30 TLOC routes
advertised via OMP advertised via OMP

NAT
31.1.2.3
Internet-1
T1
Ge0/0 T3
Ge0/1
10.1.2.3 150.2.2.2
NAT
Ge0/1 78.5.13.9 Ge0/2
T2
172.16.2.3 T4
150.2.2.2
vEdge-1
1.1.1.1 Internet-2 vEdge-2
Site-id 10 2.2.2.2
Site-id 20

T1 IPsec T3 T2 IPsec T3

T1 IPsec T4 T2 IPsec T4

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vEdge-1 INET vEdge-3

MPLS

vEdge-2 vEdge-4
LTE

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Public Color < -- > Public Color

Private Public Public Private


IP IP IP IP

NAT Internet NAT

vpn 0 vpn 0
interface ge0/0
tunnel-interface
IPsec
IPsec
interface ge0/0
tunnel-interface
color biz-internet color biz-internet
carrier default carrier default

Public Color < -- > Private Color

Private Public Public Private


IP IP IP IP

NAT Internet NAT

vpn 0 vpn 0
interface ge0/0
tunnel-interface
IPsec
IPsec
interface ge0/0
tunnel-interface
color biz-internet color mpls
carrier default carrier default

Private Color < -- > Private Color

Private Public Public Private


IP IP IP IP

NAT Internet NAT

IPsec
vpn 0 vpn 0
interface ge0/0 interface ge0/0
tunnel-interface tunnel-interface
color mpls color mpls
carrier default carrier default

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vManage vSmart vBond vEdge-2

Control Plane
Private IP

Public
IP

Internet MPLS
The private IP is NATed
to a publicly routable IP
before passing through
the Internet Private IP

vEdge-1

NAT NAT
IP IP
MPLS MPLS
(Carrier1) (Carrier2)
Overlay tunnels are built
between the NATed IP
Private IP addresses Private IP

vpn 0 vpn 0
interface ge0/0 interface ge0/0
tunnel-interface tunnel-interface
color mpls color mpls
carrier carrier1 carrier carrier2
vEdge-1 vEdge-2

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
9.9.9.9
TLOC routes Site-id 30 TLOC routes
advertised via OMP advertised via OMP

NAT
31.1.2.3
Internet-1
T1
Ge0/0 T3
Ge0/1
10.1.2.3 150.2.2.2
NAT
Ge0/1 78.5.13.9 Ge0/2
T2
172.16.2.3 T4
150.2.2.2
vEdge-1
1.1.1.1 Internet-2 vEdge-2
Site-id 10 2.2.2.2
Site-id 20

T1 IPsec T3 T2 IPsec T3

T1 IPsec T4 T2 IPsec T4

NAT
Internet-1
T1
Ge0/0 T3
Ge0/1

Ge0/1 Ge0/2
T2 T4
vEdge-1
1.1.1.1 MPLS vEdge-2
Site-id 10 2.2.2.2
Site-id 20
T1 IPsec T3 T2 IPsec T4

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vEdge-1 vEdge-2
Tunnel
metro-ethernet
Group 1

Tun mpls
Group 1
MPLS
Tunnel
mpls
Group 1

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vEdge-1 vEdge-2
Tun mpls
Group 2 Tun mpls
Group 2

MPLS
biz-internet
Tun Group 1 biz-internet
Tun Group 1

INET

public-internet
Tunnel Group 1 metro-ethernet
Tunnel Group 2

vEdge-3

Hub Site
No Tunnel Group

INTERNET

Tunnel Group 20
10

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
STUN Binding
Request
vEdge vBond

NAT Internet
Embedded Acting as a
STUN client STUN server
STUN Binding Response
Private IP “Your Public IP/Port is ... ”
Public IP

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Src X/8001
Full-Cone Dst B/80

Src A/8001 t Port 80


ke
Dst B/80 Pa
c IP-B
i t ial Port 81
In

NAT
IP-A Port 8001
IP-X
Port 80
IP-C
Port 81

Src X/8001
Restricted-Cone Dst B/80

Src A/8001 et Port 80


ck IP-B
Dst B/80 l Pa
ia Port 81
Init

NAT
IP-A Port 8001
IP-X
Port 80
IP-C
Port 81

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Src X/8001
Port-Restricted-Cone Dst B/80

Src A/8001 et Port 80


ck IP-B
Dst B/80 l Pa
itia Port 81
In

NAT
IP-A Port 8001
IP-X
Port 80
IP-C
Port 81

Src X/31644*
Symmetric Dst B/80

Src A/8001 t Port 80


ke
Dst B/80 lP
ac IP-B
itia Port 81
In

NAT
IP-A Port 8001
IP-X
Port 80
IP-C
Port 81

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Src X/8001
Port-Restricted-Cone Dst B/80

Src A/8001 t Port 80


ke
Dst B/80 lP
ac IP-B
it i a Port 81
In

NAT
IP-A Port 8001
IP-X
Port 80
IP-C
Port 81

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart

OMP OMP
Update Update

En
d wi cryp
pte th
n cry ey-3 ke ted
E
hk y-
Encr-Key-1 wit 1
MPLS Encr-Key-3

T1 T3
Encr-Key-4
Encr-Key-2
T2 T4

Encr INET ed
vEdge-1 with ypted rypt
Enc ey-2
vEdge-2
1.1.1.1 key k 2.2.2.2
-4 with
Site-id-10 Site-id-20

IP UDP EDS Original Packet

Encrypted

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
controller
En
cr
-K
ey

Enc
-1

-1
ey

r-K
r-K

ey-1
c
En
vEdge-1 generates T2
e c
an AES-256 key
IPs
and advertises it
to vSmart vEdge-2
T1

IP
se
vEdge-1 c
T3 vEdge-3
Overlay fabric

vEdge-1 Traffic encrypted and decrypted with key-2 vEdge-2

T1 IPsec tunnel T2

Traffic encrypted and decrypted with key-1


Generated Locally Generated Locally
Encr-key-1 Encr-key-2
Received from vSmart Received from vSmart
Encr-key-2 Encr-key-1

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Local Received
BA AB

Local Received
AB BA

AB
BA ith key
key dw vEdge-B
with r ypte
rypted Enc
Enc

vEdge-A Enc Local Received


ryp
ted
wit CA AC
hk
ey
CA

Enc
ryp
ted
wit
Local Received hk
ey
AC
AC CA

vEdge-C

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
WAN Edge device
Segmentation Paradigm
Controllers

Connected

Service
Ge0/2 Ge0/0 INET
VPN 5
Dynamic Transport
routing
VPN 0
Ge0/3
Service Ge0/1 MPLS
VPN 10

Management
Management
VPN 512 Network

Eth0

OOB

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart

INET MPLS

10.1.1.1/30 14.3.2.1/30

Default Default
10.1.1.2/30 Ge0/0 Ge0/1 14.3.2.2/30
Route Route
VPN 0

vEdge-1
System IP: 1.1.1.1
Site-ID: 10

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
VPN VPN
5 5
VPN5
VPN IPsec VPN
10 VPN10
tunnel 10
VPN33
VPN VPN
33 vEdge1 vEdge2 33

vSmart
controller

vEdge3 VPN
5
VPN VPN 0
VPN
5
VPN 0 10
VPN INET
10 VPN
vEdge1 5
VPN 0
VPN
10
vEdge2

IP UDP ESP VPN Data

02 4 63 4 ...

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Full-mesh Hub-and-spoke

VPN 1 VPN 2

Point-to-point
Custom-mesh

VPN 3 VPN 4

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Controller
Certificate

Root Root
Certificate Certificate

vManage vSmart

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco
Server

Certificates
retrieved vBond
CSRs Sent 2 3

Certs
1 4
installed
Generate
CSRs
Admin vManage

vSmart

CSRs
Cisco TAC Server
Signed
4

Certificates
Open TAC retrieved vBond
3
case CSRs Sent 2 5

Certs
1 6
installed
Generate
CSRs
Admin vManage

vSmart

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco
Server

CSR
4
Signed

Download
Manually Certificates vBond
Submit 3 5
CSRs
Generate
CSRs
1 Certs
7
installed
2
Download
CSRs
Admin vManage
6
Upload
certs
vSmart

Cisco TAC CSRs Signed Server

Manually
Submit CSRs
Open TAC
4 3
case
6 Download
Certificates vBond

Generate
CSRs
1
Certs
2 8 installed
Download
Admin CSRs
7
Upload vManage
certs

vSmart

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Enterprise
CA

CSR
7
Signed

Download
Certificates vBond
Get Root 1 6 8
Certificate Root Cert
3
installed
Upload Root
Cert
2
Generate Certs
10
CSR installed
4
Download
Admin CSR vManage
5
Upload Certs
9
vSmart

vBond

vManage
Admin

vSmart vSmart
All Cisco SD-WAN controllers
are defined explicitly
System
Hostname
Controller
Site-IDIP
vSmart
vBond1
5.5.5.1
vBond
20
vManage1
vManage
5.5.5.2
20
vSmart1
vSmart
5.5.5.3
30
vSmart2
vSmart
5.5.5.4
40

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
WAN Edge Deployment Options

Automated Provisioning Manual Bootstrap

PnP ZTP
via CLI via USB
(Plug-and-Play) (Zero-touch-provisioning)

Cisco IOS-XE Viptela vEdge devices All device IOS-XE

vEdge vManage vBond ZTP DNS DHCP

ZTP DHCP

OFF -> ON
Obtain IP
1
ztp.viptela.com
2
Get vBond
address
3
Authenticate
Controllers list
4
Authenticate
5
Join Fabric
6 Overlay
Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vBond vSmart1 vSmart2 vManage

Private IP

Public IP

Permanent sessions
with vBond
Controllers List:
vSmart1: Private/Public IP, System IP
vSmart2: Private/Public IP, System IP
vManage: Private/Public IP, System IP

vEdge

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vEdge vManage vBond Bootstrap Config

CFG

OFF -> ON PnP process looks for the


ciscosdwan.cfg file in flash
or bootable usb
1
The vEdge learns
the vBond address
and Organization-name
2
Authenticate to vBond
Get Controllers list
4
Authenticate
5
Join Fabric
6 Overlay
Fabric

vEdge vManage vBond CONSOLE

>_

OFF -> ON Net admin configures:


- System IP
- Site-ID
- Org-name
- vBond Address
- Transport VPN 0
1
Authenticate to vBond
Get Controllers list
4
Authenticate
5
Join Fabric
6 Overlay
Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vEdge
Authorized
List
Network
Admin

vBond vSmart1 vSmart2 vManage

Private IP

Public IP

Permanent sessions with vBond

Organization Name
Serial Number
vEdge
Root Certificate
Device Certificate

vEdge / vBond vSmart / vManage DNS NTP ICMP

UDP 12346 - 12445 UDP 12346 - 13065 UDP 53 UDP 123 Echo / Reply

Firewall
Layer

vEdge

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
4G/LTE

T3

IPsec + BFD

T1 T2
INET
IPsec + BFD
vEdge-1 vEdge-3

4G/LTE

T3
DTLS

T1 T2
INET
DTLS
vEdge-1 vEdge-3

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Last Resort A tunnel between T1–T3 forms only
Circuit in case that T1–T2 goes down

4G/LTE

T3
Last-resort
circuit

T1 T2
INET
IPsec + BFD
vEdge-1 vEdge-3

4G/LTE

T3
Last-resort
circuit

T1 T2
INET
DTLS
vEdge-1 vEdge-3

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
4G/LTE

T3
Last-resort
circuit

T1 T2
INET
IPsec + BFD
vEdge-1 vEdge-3

INET MPLS INET MPLS

DSL Leased Line


Switch Switch

Local Networks Local Networks

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
INET MPLS

Directly Connected

vEdge-1 vEdge-2

Local Networks

INET MPLS

vEdge-1 vEdge-2

L2 Switching

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
INET MPLS

vEdge-1 vEdge-2

L3 Routing

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
INET MPLS
NAT Advertise
T11 Subnet A Subnet B T22
T21
Ge0/0 Ge0/1

Ge0/4 Subnet A Ge0/4


Transport TLOC static 0.0.0.0/0 Transport
Extension
VPN 0 VPN 0
Ge0/5 Subnet B Ge0/5 TLOC
static 0.0.0.0/0 Extension

T12

Service Service
VPNs 1-511 VPNs 1-511

WAN Edge router 2


1

INET MPLS
Advertise
T11
10.51.1.0/30 10.50.2.1
Ge0/0
via BGP Ge0/1

vEdge-1 vEdge-2
Ge0/5 10.51.1.0/30 Ge0/5

vpn 0 vpn 0
interface ge0/5 T12 interface ge0/5
ip address 10.51.1.1/30 ip address 10.51.1.2/30
tunnel-interface tloc-extension ge0/1
encapsulation ipsec no shutdown
color mpls restrict !
!
ip route 0.0.0.0/0 10.51.1.2

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
INET MPLS
NAT
192.168.51.2 T22
toward INET
Ge0/0 Ge0/1

vEdge-1 vEdge-2
Ge0/4 192.168.51.0/30 Ge0/4

vpn 0 vpn 0
nat interface ge0/4 T21
! ip address 192.168.51.2/30
interface ge0/4 tunnel-interface
ip address 192.168.51.1/30 encapsulation ipsec
tloc-extension ge0/0 color public-internet
no shutdown !
! ip route 0.0.0.0/0 192.168.51.1

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
CENTRALIZED-CONTROL-POLICY

Lists Definition Application


HUB POLICY-1 site-list SPOKES
site-id 50 sequence 1 control-policy POLICY-1 out
match tloc
SPOKES site-list HUB
site-id 60 !
site-id 70 action accept
site-id 80
site-id 90 default-action reject

GUI
Policy Wizard vSmart
Controllers
Policy Activation

NETCONF transaction to vSmart


vManage

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vManage
GUI

NETCONF NETCONF

Local
LocalControl
Data Policy
Policy
(OSPF,
(ACLs, BGP,
QoS, etc.)
etc.)
Centralized
Centralized
App-Aware
VPN Memebership
Control
Data
Routing
Policy
Policy

Centralized Localized
Policies Policies
OMP

vSmart vEdge
Fabric
Application
Data Plane
SLAs

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco SD-WAN Policy

Centralized Policy Localized Policy


(Affects the whole overlay fabric ) (Affects a single device )

Control Plane Policy Data Plane Policy


(Affects the omp routes and (Directly affects the forwarding of
tlocs advertisements ) packets in the data plane)

Topology Application-aware Routing

VPN Membership Traffic Data

Cflowd

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Centralized
Local Egress
App-Route
IP Lookup Forwarding Policy
Policy
ACLs, Policing
SLA-based Routing

Service Transport
Side Side
1 2 3 4 5 6 7 8

Local Ingress Centralized


Queueing and
Policy Data Policy Security Policy
Scheduling
Classification and Path Selection, FW, IPS, etc.
LLQ, WRR, etc.
Marking Services

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Feature-based CLI-based
# the whole device
configuration
OMP !
vpn 512
interface eth0
AAA System ip address 1.1.1.5/24
description MGMT
no shutdown
VPN NTP !
OR ...

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Device Template (Model Type)

Feature
Basic Information
(System, AAA, OMP,
Feature Feature Logging, Archive, NTP)

Feature
Transport & Mgmt
VPN0 and VPN512
Feature Feature (WAN-facing features)

Feature
Service VPNs
VPN1, VPN2, etc
Feature Feature (LAN-facing features)

Feature
Additional Templates
(Banner, SNMP,
Policy Feature Local & Security policies)

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
EVE-NG
192.168.115.0/24

vBond vManage vSmart vEdge-1 vEdge-N


1.1.1.10 1.1.1.20 1.1.1.30 1.1.1.40 1.1.1.nn

eth1

Ge0/0 eth0 eth0 Ge0/0 Ge0/0


.10 .20 .30 .40 .nn

Transport VPN0: 10.1.1.0/24

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Device Certificate
Device Distinguished Name

Device Public Key


Root Certificate
Root CA Distinguished Name (ROOTCA.pem)
Referencing
Root CA Signature Root CA Distinguished Name
Verify Signature
Root CA Public Key

Root CA Signature
Sign
Self-sign

Root CA Private Key


(ROOTCA.key)

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco SD-WAN Policy

Centralized Policy Localized Policy


(Affects the whole overlay fabric ) (Affects a single device )

Control Plane Policy Data Plane Policy


(Affects the omp routes and (Directly affects the forwarding of
tlocs advertisements ) packets in the data plane)

Topology Application-aware Routing


(Controls OMP and TLOC route
advertisements)

Traffic Data

VPN Membership
(Controls distribution of routes of
particular VPNs to specific sites)
Cflowd

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
controllers
WAN edge devices send
routing information to
the vSmart controllers via
OMP.
OMP OMP
OMP

vEdge-1 vEdge-2

WAN edge devices do not


exchange any kind of
control plane information
Data Plane between themselves
vEdge-3

vSmart Controllers

Routing Table
TLOC Table

Centralized Policy
Inbound Outbound
The policy itself is never Policy Policy
pushed
to vEdge routers, only the
OM

results of the control policy


tes

are advertised to the vEdge


P
pda

Upd

routers via OMP.


PU

ate
OM

Overlay
Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
No Policy Configured

OMP routes
TLOC routes

vSmart
controllers Accepts Redistributes
all routes all routes

OM
tes

PU
pda

pda
PU

t
OM

es

Overlay
Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Policy Configured

OMP routes
TLOC routes

vSmart
controllers Only accepted routes Only accepted routes
are inserted into the are redistributed via
routing table OMP
Rejects all by default Rejects all by default

OM
s
ate

P
d

Up
Up

d
ate
P
OM

s
Overlay
Fabric

Centralized Policy

Name
Description

Policy Apply Policy


Lists
Definition to Site-list
Site List Name
Inbound
Prefix List Description
Outbound
VPN List
Default
TLOC List Sequence
Action
etc
Reject
Accept
Match Action

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cisco SD-WAN Policy
vManage GUI Configuration
via CLI
Policy Wizard

vSmart
controllers
The centralized policy is
pushed to the vSmart

OM
as a NETCONF
s
ate

PU
transaction
pd

p
PU

da
te
OM

Overlay
Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
The same
Other
Control Policy
sites

INBOUND OUTBOUND
to Site-1 to Site-2

OMP OM
P

P
M
O
vEdge-1
WAN
vEdge-3

Site-id 1 vEdge-2 Site-id 2

Outbound Policy control-policy PREFERENCE


sequence 1
- Does not affect vSmart’s best match route
path selection originator 1.1.1.2
- Affects only vEdge-3's best prefix-list SUBNET-1
path selection. !
- Does not affect other sites! action accept
set
preference 90 site-list SITE-2
! site-id 2
vEdge-1 default-action accept
1.1.1.1
Site 1 apply-policy
site-list SITE-2 vEdge-3
control-policy PREFERENCE out 1.1.1.3
172.16.1.0/24

T1
Site 2
(SUBNET-1)

T2
OMP Update
OMP Updates 172.16.1.0/24 via T1 (0)
T3 172.16.1.0/24 via T2 (0)
172.16.1.0/24 via T3 (90)
T4 172.16.1.0/24 via T4 (90)
Best Routes
vEdge-2
172.16.1.0/24 via T1
1.1.1.2
Site 1 172.16.1.0/24 via T2

172.16.1.0/24 via T3

172.16.1.0/24 via T4
Does not affect other Sites/vEdges

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Inbound Policy control-policy PREFERENCE
- Affects vSmart’s best-path selection. sequence 1
match route
- New best routes are selected. originator 1.1.1.2
- Only best routes are advertised! prefix-list SUBNET-1
- Affects the whole overlay fabric! !
action accept
set
preference 90
site-list SITE-1 !
site-id 1 !
vEdge-1 default-action accept
1.1.1.1
Site 1 apply-policy
site-list SITE-1 vEdge-3
control-policy PREFERENCE in
172.16.1.0/24

T1 1.1.1.3
Site 2
(SUBNET-1)

T2
OMP Update
OMP Updates 172.16.1.0/24 via T3
T3 172.16.1.0/24 via T4

T4
Best Routes
vEdge-2
172.16.1.0/24 via T3
1.1.1.2
Site 1 172.16.1.0/24 via T4

Affects all other Sites/vEdges

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
OUTBOUND Control
POLICY Policy
Does not influence
the OMP RIB on
OUTBOUND
vSmart to Site-Y

OMP Updates

Affects only devices on


sites listed in the
OMP Updates applied site-list

INBOUND
POLICY Affects the OMP
Control routing information
base on vSmart
Policy

Affects the
INBOUND whole overlay
to Site-X
fabric

OMP Updates

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Underlay Transport Overlay Fabric
6 vEdges * 1 TLOC IPsec tunnels = (6*5)/2 = 15
vEdge1 vEdge2 vEdge1 vEdge2

T1 T2 T1 T2

vEdge6 vEdge5 vEdge6 vEdge5

T6 T5 T6 T5
INET

T3 T4 T3 T4

vEdge3 vEdge4 vEdge3 vEdge4

Underlay Transports Overlay Fabric


6 vEdges * 2 TLOCs IPsec tunnels = 2*(6*5)/2 = 30

vEdge1 vEdge2 vEdge1 vEdge2

T11 T12 T21 T22 T11 T12 T21 T22

vEdge6 vEdge5 vEdge6 vEdge5


T61 T51

INET MPLS T61 T51


T62 T52
T62 T52

T31 T32 T41 T42 T31 T32 T41 T42

vEdge3 vEdge4 vEdge3 vEdge4

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Site 1 - Controllers Site 50 VPN 1 VPN 2

Data Center
vManage vBond vSmart
(Hub)
172.16.50.0/24 192.168.50.0

VRRP
eth1 ge0/0 eth1
vEdge-1 vEdge-2
50.50.50.50 50.50.50.51

VPN0-1.1.1.0/24 Ge0/0 Ge0/1 Ge0/0 Ge0/1


50.1.1.1 10.50.1.1 50.1.2.1 10.50.2.1

1.1.1.1

TLOC colors
aws color public-internet
INET MPLS
1.1.1.0/24 0.0.0.0/0
color mpls
color lte
LTE
4G
Site 60 Site 70 Site 80 Site 90
Ge0/0 Ge0/0 Ge0/1 Ge0/0 Ge0/1 Ge0/1
60.1.1.1 70.1.1.1 10.70.1.1 80.1.1.1 10.80.1.1 10.90.1.1

Ge0/3 Ge0/2 Ge0/3 Ge0/2 Ge0/3 Ge0/2 Ge0/3


Ge0/2

vEdge-3 BGP vEdge-4 BGP OSPF vEdge-5 OSPF ISIS ISIS


vEdge-6
60.60.60.60 70.70.70.70 80.80.80.80 90.90.90.90
VPN 1 VPN 2 VPN 1 VPN 2 VPN 1 VPN 2 VPN 1 VPN 2

192.168.60.0 198.18.60.0/24 172.16.70.0/24 192.168.70.0 172.16.80.0/24 192.168.80.0 172.16.90.0/24 192.168.90.0

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
GUI CLI

Policy Policy sent as Policy


NETCONF
a transaction
to vSmart Policy Applied in OUT
vSmart direction is affecting the
outgoing OMP
Advertisements

vManage
O
M
Cs e
LO at

P
l T pd

A ll T
dv L
ca U

er OC
Lo P
M

ti s
se
O

m
en
t

vEdge-1 vEdge-6
Cisco SD-WAN Overlay Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Site 1 - Controllers Site 50 VPN 1 VPN 2

Data Center
vManage vBond vSmart
(Hub)
172.16.50.0/24 192.168.50.0

VRRP
eth1 ge0/0 eth1
vEdge-1 vEdge-2
50.50.50.50 50.50.50.51

VPN0-1.1.1.0/24 Ge0/0 Ge0/1 Ge0/0 Ge0/1


50.1.1.1 10.50.1.1 50.1.2.1 10.50.2.1

1.1.1.1

TLOC colors
aws color public-internet
INET MPLS
1.1.1.0/24 0.0.0.0/0
color mpls
color lte
LTE
4G
Site 60 Site 70 Site 80 Site 90
Ge0/0 Ge0/0 Ge0/1 Ge0/0 Ge0/1 Ge0/1
60.1.1.1 70.1.1.1 10.70.1.1 80.1.1.1 10.80.1.1 10.90.1.1

Ge0/3 Ge0/2 Ge0/3 Ge0/2 Ge0/3 Ge0/2 Ge0/3


Ge0/2

vEdge-3 BGP vEdge-4 BGP OSPF vEdge-5 OSPF ISIS ISIS


vEdge-6
60.60.60.60 70.70.70.70 80.80.80.80 90.90.90.90
VPN 1 VPN 2 VPN 1 VPN 2 VPN 1 VPN 2 VPN 1 VPN 2

192.168.60.0 198.18.60.0/24 172.16.70.0/24 192.168.70.0 172.16.80.0/24 192.168.80.0 172.16.90.0/24 192.168.90.0

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
GUI CLI

Policy Policy sent as Policy


NETCONF
a transaction
to vSmart Policy Applied in OUT
vSmart direction is affecting the
outgoing OMP
Advertisements

vManage
O
M
Cs e
LO at

P
l T pd

A ll T
dv L
ca U

er OC
Lo P
M

ti s
se
O

m
en
t

vEdge-1 vEdge-6
Cisco SD-WAN Overlay Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
CENTRALIZED-CONTROL-POLICY-V2

Lists Definition Application


HUB FILTER-TLOCS-AND-ADV-ROUTES site-list SPOKES
site-id 50 sequence 1 control-policy FILTER-TLOCS-AND-ADV-ROUTES out
match tloc
SPOKES site-list HUB
site-id 60 !
site-id 70 action accept
site-id 80
site-id 90 sequence 11
match route
prefix-list ALL-ROUTES
!
action accept
!
default-action reject

GUI
Policy Wizard

Policy Activation

NETCONF transaction to vSmart


vManage vSmart
Controllers

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
CENTRALIZED-CONTROL-POLICY-V2

Lists Definition Application


HUB FILTER-TLOCS-AND-ADV-ROUTES site-list SPOKES
site-id 50 sequence 1 control-policy FILTER-TLOCS-AND-ADV-
match tloc ROUTES out
SPOKES site-list HUB
site-id 60 !
site-id 70 action accept
site-id 80 !
site-id 90 sequence 11
match route
prefix-list ALL-ROUTES
!
action accept
!
default-action reject

Centralized Control Policy applied


in out direction to the spokes
vSmart
that affects the OMP
Controller
advertisements

vEdge-1 vEdge-2 vEdge-3 vEdge-4 vEdge-5 vEdge-6


site-id 50 site-id 50 site-id 60 site-id 70 site-id 80 site-id 90

HUB SPOKES

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Site 1 - Controllers Site 50 VPN 1 VPN 2

Data Center
vManage vBond vSmart
(Hub)
172.16.50.0/24 192.168.50.0

VRRP
eth1 ge0/0 eth1
vEdge-1 vEdge-2
50.50.50.50 50.50.50.51

VPN0-1.1.1.0/24 Ge0/0 Ge0/1 Ge0/0 Ge0/1


50.1.1.1 10.50.1.1 50.1.2.1 10.50.2.1

1.1.1.1

TLOC colors
aws color public-internet
INET MPLS
1.1.1.0/24 0.0.0.0/0
color mpls
color lte
LTE
4G
Site 60 Site 70 Site 80 Site 90
Ge0/0 Ge0/0 Ge0/1 Ge0/0 Ge0/1 Ge0/1
60.1.1.1 70.1.1.1 10.70.1.1 80.1.1.1 10.80.1.1 10.90.1.1

Ge0/3 Ge0/2 Ge0/3 Ge0/2 Ge0/3 Ge0/2 Ge0/3


Ge0/2

vEdge-3 BGP vEdge-4 BGP OSPF vEdge-5 OSPF ISIS ISIS


vEdge-6
60.60.60.60 70.70.70.70 80.80.80.80 90.90.90.90
VPN 1 VPN 2 VPN 1 VPN 2 VPN 1 VPN 2 VPN 1 VPN 2

192.168.60.0 198.18.60.0/24 172.16.70.0/24 192.168.70.0 172.16.80.0/24 192.168.80.0 172.16.90.0/24 192.168.90.0

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
VPN1
172.16.50.0/24

vEdge-1 vEdge-2
50.50.50.50 50.50.50.51

to vSmart T1 T2 T3 T4 to vSmart

172.16.50.0/24 via T1 172.16.50.0/24 via T3


172.16.50.0/24 via T2 172.16.50.0/24 via T4

INET MPLS
from from
vSmart vSmart

T5 T6

vEdge-4
70.70.70.70

VPN1
172.16.70.0/24

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart Controllers

Routing Table
TLOC Table

Centralized Policy
Inbound Outbound
The policy itself is never Policy Policy
pushed
to vEdge routers, only the
OM

results of the control policy


s
ate

are advertised to the vEdge


PU
Upd

routers via OMP.


pd
ate
P
OM

Overlay
Fabric

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Security
Stack

vEdge-3
Site-3

T3

vEdge-1 vEdge-2

T1
WAN T2

Site-1 Site-2

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Security
Stack

vEdge-3
Site-3

T3
Tunnel T2-T3
goes down
vEdge-1 vEdge-2

T1
WAN T2

Site-1 Site-2

Security
Stack

Intermediate vEdge-3
Router Site-3

T3
Ultimate
TLOC
Ultimate
vEdge-1 Destination
vEdge-2

T1 WAN T2

Site-1 Site-2

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Normal Operations In case of failure (STRICT OPTION)

T3 T3

INET INET
T1 T2 T1 T2

Normal Operations In case of failure (PRIMARY OPTION)

T3 T3

INET INET
T1 T2 T1 T2

Normal Operations In case of failure (BACKUP OPTION)

T3 T3

INET INET
T1 T2 T1 T2

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Normal Operations In case of failure (ECMP OPTION)

T3 T3

ECMP
INET INET
T1 T2 T1 T2

Control Policy

172.18.2.0/24 via T3 172.18.1.0/24 via T3


TLOC Action: [Strict, TLOC Action: [Strict,
Primary, Backup, ECMP] Primary, Backup, ECMP]

Site-3 vEdge-3
3.3.3.3

T3

vEdge-2
Site-1 vEdge-1
1.1.1.1 2.2.2.2
Site-2
INET
T1 T2

172.18.1.0/24 172.18.2.0/24

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Guest VPN2

vEdge-4

vEdge-3
SD-WAN
Overlay Guest VPN2
Fabric vEdge-5

vEdge-6

Guest VPN2
Guest VPN2

sequence 1
match vpn 1,3-65000
VPN Membership
action accept Policy
!
default action reject
OUTBOUND vEdges do not receive
to site-list X any routing information
VPN
associated with VPN2
MEMBERSHIP
POLICY OMP Updates

vRoutes for VPN1 site-list X


vSmart vRoutes for VPN2
vRoutes for VPN3
...

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Site 1 - Controllers Site 50
vManage vBond vSmart Data Center
192.168.50.0/24
(Hub)
VPN2 VRRP VPN2
eth1 ge0/0 eth1
vEdge-1 vEdge-2
50.50.50.50 50.50.50.51
VPN0-1.1.1.0/24 Ge0/0 Ge0/1 Ge0/0 Ge0/1
50.1.1.1 10.50.1.1 50.1.2.1 10.50.2.1

TLOC colors
aws color public-internet
INET MPLS
1.1.1.0/24 0.0.0.0/0
color mpls
color lte

LTE
4G
Site 60 Site 70 Site 80 Site 90
Ge0/0 Ge0/0 Ge0/1 Ge0/0 Ge0/1 Ge0/1
60.1.1.1 70.1.1.1 10.70.1.1 80.1.1.1 10.80.1.1 10.90.1.1

vEdge-3 vEdge-4 vEdge-5 vEdge-6


60.60.60.60 70.70.70.70 80.80.80.80 90.90.90.90

VPN2 VPN2 VPN2 VPN2

192.168.60.0/24 192.168.70.0/24 192.168.80.0/24 192.168.90.0/24

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
From-Service
Overlay
Fabric

INET 4G/LTE

MPLS
From-Tunnel

VPN 3
1
2

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vManage
GUI

NETCONF NETCONF

vManage

Local
LocalControl
Data Policy
Policy
(OSPF,
(ACLs, BGP,
QoS, etc.)
etc.)
Centralized
Centralized
App-Aware
VPN Memebership
Control
Data
Routing
Policy
Policy

Centralized Localized
Policies Policies
OMP

vEdges
Fabric
Application
Data Plane
SLAs
vSmart

Apply Policy
Policy
to
Definition
Site-list

List of sites List of Prefixes Policy Instance

Sequence Default Action

Action
Match

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
1
Define an AAR policy
vManage vSmart that matches
applications to SLA
NETCONF
requirements

2
Push policy to vEdges
SD-WAN
fabric
3
Measure packet loss,
latency and jitter of
overlay tunnels
Site-list
VPN-list VPN1 VPN2 VPN3
App-route Policy Mapp applications to
tunnels based on SLA
4 SLA-class
Packet loss <=2%
Latency <=200ms
App4
App3
App2
App1 Jitter <= 25ms

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
n1
sla-class VOICE-SLA
sequence 11
match dscp 46
loss 2 Tu
latency 200
!
action
jitter 50 Metro-Eth
sla-class VOICE-SLA
2
Tun
metro-eth
Data traffic MPLS
with dscp 46 AAR mpls

Policy lte Tun 3

biz-inet 4G/LTE
Tu
n4
Data traffic is ECMP
forwarded across all colors
that meet the SLA
INET

sla-class VOICE-SLA
sequence 11 loss 2 n1
match dscp 46 latency 200 Tu
!
action
jitter 50 Metro-Eth
sla-class VOICE-SLA preferred-color mpls

2
Tun
metro-eth
Data traffic MPLS
with dscp 46 AAR mpls

Policy lte Tun 3

biz-inet 4G/LTE
Tu
n4
Data traffic is pinned to the
mpls color while it meets SLA INET

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
sla-class VOICE-SLA
sequence 11 loss 2
n1
match dscp 46
!
latency 200 Tu
jitter 50
action Metro-Eth
sla-class VOICE-SLA preferred-color mpls lte

2
Tun
metro-eth
Data traffic MPLS
with dscp 46 AAR mpls

Policy lte Tun 3

biz-inet 4G/LTE
Tu
n4
Data traffic is ECMP forwarded
across all preferred colors
that meet the SLA INET

sla-class VOICE-SLA
sequence 11 n1
match dscp 46
loss 2
latency 200
Tu
! jitter 50 Metro-Eth
action
sla-class VOICE-SLA strict
2
Tun
metro-eth
Data traffic MPLS
with dscp 46 AAR mpls

Policy lte Tun 3

biz-inet 4G/LTE
Tu
n4
Data traffic is dropped
if no color meet the SLA INET

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
sla-class VOICE-SLA
loss 2 y
1 nc
latency 200 n ate
sequence 11 jitter 50 Tu st l
match dscp 46 fallback-best-tunnel e
! criteria latency s low Metro-Eth
action ha
sla-class VOICE-SLA fallback-to-best-path
2
Tun
metro-eth
Data traffic MPLS
with dscp 46 AAR mpls

Policy lte Tun 3

biz-inet 4G/LTE
Tu
n4
Data traffic is pinned to the
best color based on criteria
defined in the SLA-class INET

sla-class VOICE-SLA
sequence 11 loss 2
match dscp 46 n1
!
latency 200
jitter 50
Tu
action Metro-Eth
sla-class VOICE-SLA preferred-color mpls lte
backup-sla-preferred-color metro-ethernet
2
Tun
metro-eth
Data traffic MPLS
with dscp 46 AAR mpls

Policy lte Tun 3

biz-inet 4G/LTE
Tu
n4
Data traffic is pinned to the
backup preferred color if no
color meet the SLA INET

sla-class DEFAULT-SLA
sequence 11
loss 5
match dscp 46
latency 300 1
! n
action
jitter 50 Tu Metro-Eth
sla-class VOICE-SLA preferred-color mpls
!
default-action sla-class DEFAULT-SLA 2
Tun
metro-eth
MPLS
AAR mpls

Policy lte
Tun 3

biz-inet 4G/LTE
Tu
n4
Non matching data traffic is
ECMP forwarded across all colors
Non-matching
traffic that meet the default SLA INET

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Application-Aware Routing (AAR) Packets match AAR Sequence
Tunnel Selection Flow

sla-class
NO
configured?

(Only one can


YES
be configured)
One is configured? NO
Fallback-to-best-path YES Tunnels
NO Strict configured NO meeting SLA?
Backup-preferred-colors
YES YES
YES
Drop the packets Preferred-color
NO
configured?

YES

Backup-preferred- Preferred-color
YES colors down?
NO NO down? YES

ECMP on default ECMP on default SLA Send the ECMP on tunnels ECMP on tunnels
SLA and all and backup-preferred packets using meeting SLA and meeting SLA and
colors colors best color preferred colors all colors

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
SaaS
Traditional Applications
WAN Model
INET
Users Data center,
Branch
Regional Hub

WAN Security
Stack

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Office 365
Applications

office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

Regional
DNS DNS Hub

ISP-1 ISP-3
DNS
DNS
ISP-2
ISP-4

SD-WAN
fabric
Branch-1
Data Center

Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

DNS resolution for


configured SaaS apps
over each ISP circuit Regional
DNS DNS Hub
Periodic HTTPs pings
over each ISP
ISP-1 ISP-3
DNS
DNS
2 ISP-2
ISP-4
1
User SD-WAN
fabric
Branch-1

Data Center

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

DPI engine intercepts


user DNS queries for
SaaS apps Regional
DNS DNS Hub
DNS queries for non-
SaaS apps are not
ISP-1 ISP-3
intercepted
DNS
DNS
ISP-2
ISP-4
3
User 4 SD-WAN
fabric
Branch-1

Data Center

Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

Cloud OnRamp routes


the users’ flows over Regional
the best performing
DNS Hub
path
ISP-1 ISP-3

DNS
ISP-2
ISP-4
5 Latency /
Packet loss
User SD-WAN
fabric
Branch-1

Data Center

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

DNS resolution for


configured SaaS apps over
the local ISP circuit Regional
DNS Hub
DNS resolution over the
gateway vEdge router DNS ISP-3
ISP-1
DNS

ISP-4

User SD-WAN
fabric
Branch-1

Data Center

Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

Quality probing using


HTTPs pings
Regional
DNS Hub
Overlay Tunnel from Latency
the SD-WAN fabric Packet loss ISP-3
ISP-1
DNS

ISP-4

User SD-WAN
Adve
rt fabric
Branch-1 prob ises the
e valu HT
es ov TPs
er OM
Data Center P

Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

DNS resolution for


configured SaaS apps over
the local ISP circuit Regional
DNS Hub
DNS resolution over the
gateway vEdge router DNS ISP-3
ISP-1
DNS

ISP-4

User SD-WAN
fabric
Branch-1

Data Center

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

Regional Regional
Hub-1 ISP-2 ISP-3 Hub-2

ISP-4

User SD-WAN
fabric
Branch-1

Data Center

Office 365
Applications
office.microsoft.com sharepoint.microsoft.com
teams.microsoft.com

SWG SWG
POP-1 POP-2

ISP-1
ISP-2

User SD-WAN
fabric
Branch-1

Data Center

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
vManage
vBond

MPLS aws
Cloud
INET
Data Center
Azure
Cloud

Regional Hub Branch

Campus

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Cloud credentials IaaS instances are
vManage are added to mapped to VPN
vManage segments

IaaS Instance 1
INET

IaaS Instance 2
MPLS
vEdges Overlay Fabric
Cloud Provider
Region 1 vManage instantiate
vEdge instances in users
accounts and connects
IaaS instances to vEdge
GW VPN segments

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Controllers
AWS Region

VGW
AZ1
BGP<->OMP

INET AZ2
vEdge
Host VPC

vEdge
MPLS AZ1
Direct
vEdges
Connect IPsec + BGP
VGW
AZ2
SD-WAN Fabric Host VPC

Transit VPC Host VPC


BGP ASN X BGP ASN Y

IPsec tunnel

IPsec tunnel
VPN
vEdge Connection

IPsec tunnel VPN


Gateway
(VGW)
IPsec tunnel
VPN
vEdge Connection

One Elastic IP address corresponding Two IPsec tunnels


Two BGP peers per
to the transport interface(VPN0) on per AWS Site-to-
vEdge. Each corresponds
each vEdge serves as the AWS Site VPN
to the AWS-side of the
Customer Gateway (CGW) for AWS
logical tunnel
Site-to-Site VPN

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
Controllers
Azure Region VNET
VPN
GW
AS1
VNET BGP<->OMP

AS2
INET vEdge
Host VNET
VNET

vEdge
MPLS AS1
Express
vEdges
Route IPsec + BGP
VPN
GW
AS2
SD-WAN Fabric Host VNET

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
vManage
vBond
ls
nne AZ1
Tu
PN
ecV
s
IP
vEdge
SD-WAN AZ2
Tunnels

MPLS Host VPC


vEdge
Data
Center
INET IPs
SD-WAN ec
VP AZ1
Campus Tunnels vEdge NT
un
n els
Cisco SD-WAN Transit VPC
Overlay Fabric AZ2

Host VPC

vSmart
vManage
vBond

Branch
vEdge ISP-1 AWS AZ1
AWS vEdge
Network
ISP
AZ2
LAN ISP-2
Host VPC
vEdge
Cisco SD-WAN Transit VPC
Overlay Fabric
Client

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
vManage
vBond

AZ1

vEdge
SD-WAN AZ2
Tunnels IPse
Tunn c
els
MPLS Host VPC
vEdge
Data
Center Transit
INET Gateway
SD-WAN (TGW) AZ1
Campus Tunnels vEdge
Cisco SD-WAN Transit VPC
Overlay Fabric AZ2

Host VPC

vSmart
vManage
vBond
WAN/Event Telemetry
AZ1

vEdge
SD-WAN TGW
con AZ2
Tunnels nec
t
MPLS Host VPC
vEdge
Data
Center Transit
INET Gateway
SD-WAN (TGW) AZ1
Campus Tunnels vEdge
Cisco SD-WAN Transit VPC
Overlay Fabric AZ2

Host VPC

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
vSmart
vManage
vBond

AZ1

vEdge
AZ2
IPsec
VPN Tunnels
MPLS Host VPC

IPsec
Data ls
VPN Tunne
Center
INET AWS Transit
Gateway AZ1
Campus (TGW)
Cisco SD-WAN
Overlay Fabric AZ2

Host VPC

For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954
For details contact: www.networkershome.com | info@networkershome.com | Mob: +91 9611027980 | +91 9354284954

You might also like