Contrail SD-WAN Solution Design Guide

JUNIPER NETWORKS CONFIDENTIAL
CONTRAIL SD-WAN SOLUTION DESIGN GUIDE

Contrail SD-WAN Solution Guide
Contents
Contrail SD-WAN Solution Design Guide......................................................................................................................................................1
Document Version Control ...............................................................................................................................................................................5
Version History ................................................................................................................................................................................................5
Introduction...........................................................................................................................................................................................................6
SD-WAN Deployment Considerations ...........................................................................................................................................................7
Juniper Recommendation ..............................................................................................................................................................................8
Contrail SD-WAN Solution Overview ............................................................................................................................................................9
CSO Service Types ..............................................................................................................................................................................................9
Topology Components .................................................................................................................................................................................... 10
Enabling Multi-Tenancy and Segmentation. .......................................................................................................................................... 10
CSO Overlay (Tunnels) Network............................................................................................................................................................... 12
Operations and Maintenance (OAM) ....................................................................................................................................................... 13
Juniper Recommendation ....................................................................................................................................................................... 13
Mesh Tags ...................................................................................................................................................................................................... 14
Use Cases ................................................................................................................................................................................................... 14
Controller Based on-demand Mesh Dynamic Virtual Private Network (DVPN) ........................................................................... 15
Local Breakout Options -FIX...................................................................................................................................................................... 16
Breakout Profile and Breakout Policy Management ............................................................................................................................ 17
Juniper Recommendation ........................................................................................................................................................................... 18
APBR and SLA Management...................................................................................................................................................................... 18
Service Level Agreement (SLA) Policy and Profiles. ............................................................................................................................. 18
Configuration Templates ............................................................................................................................................................................ 19
Device Types ..................................................................................................................................................................................................... 21
Hub Devices................................................................................................................................................................................................... 22
Provider Hubs ........................................................................................................................................................................................... 22
Enterprise Hubs ........................................................................................................................................................................................ 22
Spoke Devices ............................................................................................................................................................................................... 24
On-premise Spoke.................................................................................................................................................................................... 24
Cloud Spoke............................................................................................................................................................................................... 25
Device Deployment Overview. ..................................................................................................................................................................... 25
JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 2

Active/Active or Active/Backup ........................................................................................................................................................... 26

Scaling ............................................................................................................................................................................................................. 27
Factors that Affect Scale ........................................................................................................................................................................ 27
BGP Sessions Supported ........................................................................................................................................................................ 28
IPsec performance ................................................................................................................................................................................... 29
DVPN .......................................................................................................................................................................................................... 30
Additional Bandwidth Requirements ................................................................................................................................................... 30
Juniper Recommendation ........................................................................................................................................................................... 31
Deployment Considerations........................................................................................................................................................................... 31
Public vs. Private IP Addresses .................................................................................................................................................................. 31
IP Services ...................................................................................................................................................................................................... 32
Underlay (Physical) Network...................................................................................................................................................................... 32
Logging ............................................................................................................................................................................................................ 33
Provider Hub Location Options ................................................................................................................................................................ 33
Enterprise Hub Locations ........................................................................................................................................................................... 34
Spoke Site Configuration Options ............................................................................................................................................................ 34
WAN Interface Types – OAM and Data ............................................................................................................................................. 34
Spoke Configuration Options .................................................................................................................................................................... 35
Primary Hub Site ...................................................................................................................................................................................... 35
Multihoming .............................................................................................................................................................................................. 35
Enterprise Hubs ........................................................................................................................................................................................ 36
IP Prefix (for Loopback IP Address)...................................................................................................................................................... 36
Local Breakout .......................................................................................................................................................................................... 36
Other Options ........................................................................................................................................................................................... 36
Intent Based policies .................................................................................................................................................................................... 37
Firewall Policies............................................................................................................................................................................................. 37
NAT .................................................................................................................................................................................................................. 37
SLAs ................................................................................................................................................................................................................. 37
Configuration Templates ............................................................................................................................................................................ 38
Preparing for Deployment .............................................................................................................................................................................. 38

Determine the Deployment Type ............................................................................................................................................................. 38

Create Comprehensive Design Document ............................................................................................................................................. 39
On-Premise Installation .......................................................................................................................................................................... 40
Site Readiness ........................................................................................................................................................................................... 40
Conclusion .......................................................................................................................................................................................................... 40
Terminology ....................................................................................................................................................................................................... 41

Document Version Control

Author: Andy Lupher
Version: Version 1v31
Date: July 17, 2020
Version History
Version Number Author Date Reason for Change
1.0 Andy Lupher 5-02-19 • Migrate 4.1 doc to incorporate 5.0
1.v5 Mark Scherbring 5-06-19 • Comments and additional material
1v31 Andy Lupher 7-17-20 • Updated for 5.3 CSOaaS

Introduction
Software-defined wide-area network (SD-WAN) is an automated, programmatic approach to managing enterprise network
connectivity and circuit costs. It extends software-defined networking (SDN) into an application that businesses can use to
quickly create a smart “hybrid WAN” – a WAN that comprises business-grade IP VPN, broadband Internet, and wireless
services. Hybrid WAN architectures enable companies to manage their growing number of applications, particularly when
using the cloud. Traffic is automatically and dynamically forwarded across the most appropriate and efficient WAN path
based on network conditions, the security and quality-of-service (QoS) requirements of the application traffic at hand, and
cost of the circuit. The enterprise customer sets the routing policies describing the intent of how they want traffic to be
routed
Organizations often review their WAN capabilities, either to add capability (features), increase availability, or to reduce
costs. Some automation of router configurations, for network tasks, is possible by Network Management Systems but fully
automating branch deployment, policy and monitoring reduces man hours and eliminates unforced errors. This differs from
legacy NMS in that active management and monitoring of links can optimize link usage and application performance via
Juniper Networks’ Contrail SD-WAN solution.
Juniper Networks’ Contrail SD-WAN solution delivers a simple, automated Multicloud platform for life-cycle support of
your branch location as well as extending into your Virtual Private Clouds. It enables you to create an evolvable
architecture to simplify growth, from secure routers to SD-WAN and SD-Branch. It automates the WAN edge across
Juniper cloud endpoints and on-premise next-generation firewalls or universal CPE platforms. Juniper Networks’ Contrail
SD-WAN solution provisions and enforces multilevel security policy at scale, across multiple clouds and enterprise sites
with diverse topologies. It provides a solution to provision, configure, monitor and report on the environments ensuring
you meet your branch network operational requirements.
Contrail Service Orchestration is the application, which is used to design, secure, automate, and manage the SD-WAN
service lifecycle across say Juniper Networks’ branch portfolio of devices within the Contrail SD-WAN environment. CSO
is available as an on-premise distribution and also as a Software as a Service (SaaS) beginning with the CSO 5.0 release. Not
all features are the same between the two releases so we will note any differences in this paper.
Contrail SD-WAN is the foundation on which you can chart a course to SD-Branch and beyond, seamlessly integrating
provisioning, full-stack security, and monitoring. Deployments can be onboarded and then monitored and configured from
a single pane of glass to reduce operational complexity. Granular Role Based Access Control allows for organizational
responsibility to stay where it belongs within your organization or crafted to meet organizational change that may occur.
Figure1 depicts the SD-WAN network topology that allows for greater management control than has been possible in the
past.
This paper provides guidance and examples to assist with design considerations of Juniper Networks’ Contrail SD-WAN
Solution, which expands on the Contrail Service Orchestration (CSO) installation and deployment guides.

Figure 1 Topology view over SD-WAN Overlay
SD-WAN Deployment Considerations

In preparation for deployment, of an SD-WAN solution, you need to consider current and long-term needs for your
environment by considering the following questions you can optimize a model that meets your branch location
requirements. These questions will assist with selecting the right application, deployment model and sizing of the servers,
Hubs, CPE devices and routers correctly.
Juniper Networks Contrail Service Orchestration provides the tools to create different topologies depending on tenant and
site requirements. Once the tenant / site requirements are understood then we can start working through deployment
considerations and designs.
Please consider the following questions when evaluating branch networking requirements:
What is the motivation for SD-WAN?
Increased bandwidth at lower cost
Unable to buy a single link with required capacity
increase functionality?
increased availability?
Are you a managed services customer?
What types of branch CPE devices are needed?
What network services run at the branch?
How do you currently manage and operate the branch network?
How do you order WAN links to branch’s?
Do you need dynamic policy-based routing over multiple links?
Do you run cloud-based service such as Microsoft Office 365, Salesforce.com, Slack, GitHub etc.?

Do you need to connect to AWS VPC or other cloud provider?
Do you need Application Aware Security?
Does your CPE choose best path based on application?
How will you prefer to monitor the SD-WAN network?
What reporting functions do you required?
Once an adoption decision has been made the next question is whether to use a cloud-based service, SP provided service
or on-premise managed service. We have included the following flow chart to help guide the decision.
Yes
DIY on premises
Large
A lot Size of network

Small
organization
Enterprise Purchase or Rent Yes Few

No. of branches
Cloud Delivered
No
Branches across Large
multiple SP A lot Size of network
globally organization
Small
No. of branches
No
SP Delivered
Few
© 2018 Juniper Networks J3.02.P01.T07 Rev 17 Figure 2 Decision Flow Chart

Template Owner: Radhika Narayanan © 2017 Juniper Networks, Inc. All rights reserved.
In general, the number of sites, geographic location, and the availability of in-house talent will be the determining factors
with regards to CSO as a service or as an On-Premise implementation. A business with less than 200 locations and existing
WAN links should look at a cloud delivered solution. Enterprises that are migrating to new service providers or need
assistance with integration and migration should look at an SP delivered platform. Large deployments with in-house
personnel would probably opt to install an On-Premise solution and customize as needed.
CSO Cloud Delivered or Software as a Service (SaaS) provides standardized templates for CPE and HUBs as it is geared
towards a prescriptive approach to reduce the margin for error. End-point customers will know exactly what ports need to
be used for WAN and LAN links and what configuration is available.
Juniper Recommendation
Customers requiring 200 or less branch sites and standard configurations should look at CSO Software as a Service.
Customers with more than 200 sites or customers that require customized configurations should use the On-Premise
solution or SP/Integrator delivered solution. A customized configuration would be where a customer wants to use
specialized templates (known as a Configuration Template) to a device. For instance, they may want the option to add a
template for specific routes per device or add RADIUS settings to an endpoint.

Contrail SD-WAN Solution Overview

Juniper Networks’ Contrail SD-WAN solution offers a fully automated end-to-end provisioning of CPE devices. The
automated workflow includes:
1. Zero touch device provisioning (ZTP) and cloud-based device management.

2. Provisioning enterprise WAN routing including VPN (OTT or native SP provided VPN service) and across internet
links. The enterprise WAN connectivity supports multiple concurrent private or public links. SD-WAN requires
that the VPN end points provisioning is fully automated and may include application aware routing and application
SLA management. When the branch site has multiple WAN links (SP managed VPN and or IPsec over internet
VPN) it is important that
a. The links are appropriately utilized
b. Efficient load distribution of traffic across links
c. WAN links have different SLA characteristics and it is important to match application flows to appropriate
links based on application SLA requirements.
3. Controller based Dynamic-VPN to establish site-to-site tunnels when sites are not behind Symmetric NAT.
4. CSO can provision LAN configuration on CPE/EX including DHCP or static addressing, VLANs, and DNS servers.
5. Firewall, NAT and UTM intents can be applied to allow traffic forwarding and block access as needed.
6. Any configuration not applied by CSO can be applied with Configuration Templates.
7. Once a site is on-boarded CSO can monitor status and provide detailed analytics and reporting on a per-site basis.
8. Active Monitoring and configuration management of the branch edge device to add service functionality and
additional configuration.
Supporting devices that create the SD-WAN overlay environment are certified by Juniper Networks’ System Test team and
are fully supported by JTAC, although other devices may work as well but outside the scope of this Design Guide. Please
read the guidelines for Hub selection to decide which devices and where to deploy. Devices can be deployed with multiple
WAN interfaces consisting of MPLS type circuits, a better definition would be “private line” and “Internet” which is best-
effort type service. Internally CSO will tag traffic types as either MPLS or Internet but they should be thought of as just
different CSO levels rather than an LDP/RSVP driven MPLS circuit with all the accompanying features and knobs.
CSO Service Types

CSO supports Standalone Next Generation Firewalls (NGFW) and SDWAN configured devices. All Service Types are supported
within a single tenant simultaneously. When creating a tenant, you now enable services such as SD-WAN, Next Gen Firewall and
then add services as needed with site and configuration templates.
NGFW can be used for a managed router, which is industry standard today but allows for centralized provisioning and monitoring
with the potential to move to a full SD-WAN branch in the future. NGFW do not require Hubs for Overlay tunnels or connectivity
or modification to the existing network infrastructure. Once imported into CSO they can be configured via UI or Configuration
templates. SRX devices that are currently installed in a network can be easily imported to CSO for monitoring and further
configuration with minimal impact on existing configuration.
SDWAN builds on existing network topology and physical paths to allow over the top tunnels to create virtual topology on
top of physical topology. It monitors links for availability and can move traffic based on link conditions to a more suitable
link. It does require the addition of Hubs to terminate the overlay tunnels and availability of routable IP addresses for those
hubs.

Topology Components
The type of site and the functionality required determine which topology components will be implemented by CSO per
site. NGFW refers to independent devices with routing, switching and security features that optionally have overlay
tunnels but only have a connect to OAM hubs for monitoring and management.
SDWAN Spokes have overlay tunnels connection to one or more Provider and/ or Enterprise hubs, in addition, mesh tags
can be used to create additional topologies to direct traffic based on geographic, redundancy or to balance traffic. If you
decide to use an SD-WAN approach in your deployment then Spokes will have a data path provisioned to a Provider
and/or Enterprise Hub as an anchor point and all non-local flows will be directed to the anchor point via routes learned
from the Virtual Route Reflector within CSO. The anchor points terminate overlay tunnels and provide forwarding to
underlay.
SDWAN Sites will default to a Hub and Spoke topology until either a system or user defined threshold for sessions is met and then
sites may connect directly to other sites if Spokes (depending on NAT Type). Each site within a tenant will establish a tunnel to Hubs
for Operations and Maintenance (OAM) and then to Enterprise Hub (if given). If no Enterprise Hub is specified, then only Provider
Hub connections will be used. Any site can be configured for a variety of options that will determine forwarding by application, site,
segment or department. Provider Hub redundancy is enabled via multi-homing where Provider Hubs are selected as primary and
secondary respectively. Hub and Spoke is useful when CPE sites are behind NAT or if there is a governance policy for all sites traffic
to pass through a central point for security scanning. In that case the number of active sessions per site can be set so no site to site
VPNs are established.
Sites that meet session criteria and routable IP addresses have the ability to dynamically establish site-to-site tunnels and mesh as
needed. All Spokes require an OAM HUB and at least one Provider Hub or Enterprise Hub for anchor points and for provisioning
and monitoring. Spokes should be connected to two hubs to ensure a backup data path in the event that one hub is not available.
Dynamic VPN tunnels for site-to-site connectivity are established based on the number of sessions between any pair of sites.
Dynamic mesh allows tunnels to be constructed as needed and torn down when traffic between sites hit a lower threshold defined
in CSO policy. Dynamic VPN tunnels between sites offloads the Provider Hubs and creates flexible topologies. Spokes with public
IP addresses can directly establish site-to-site connections or may use hole-punching for site-to-site connections, otherwise the
Spokes will use their configured Hub sites. Use Table 1 to determine the logical topology for sites and tenants.
While configuring the site, if one or more WAN links have mesh tags that match associated spokes within the same tenant
then dynamic VPNs may established with those sites as required. If mesh tags or D-VPN establishment metrics are not met
than the site is connected only to Hubs as if in a Hub-Spoke topology. Thus, a single tenant can support both Hub-Spoke
and Dynamic mesh within their SD-WAN environment.
Enabling Multi-Tenancy and Segmentation.

The Contrail SD-WAN solution uses Virtual Route Forwarding tables (VRFs) (implemented as routing instances in the Junos
OS) and MP-BGP route targets to provide tenant route separation and enable multi-tenancy. This allows segmentation by
tenant or department and assists with scaling the deployment and applying intent based policies at tenant, site or
department level.
These concepts can be illustrated using an L3VPN environment as an example. As shown below, each customer is assigned
a unique route target value, and all sites of the customer VPN use that route target value. When a router advertises a
customer’s routing information it attaches the appropriate route target value based on which customer VRF originated the
advertisements. The receiving router uses the attached route target value to identify the customer VRF into which the
received routing information should be placed.

Figure 3 Network Slicing for Multi-Tenancy
A Hub-and-Spoke environment uses route targets differently, as shown below. For each customer, every Spoke VRF
attaches the same route target value when sending routing information. The receiving Hub accepts routes with that same
route target value and installs them into Data Hub VRF. By contrast, the Data Hub VRF attaches a different route target
value when sending routing information, and the receiving Spokes accept and install routes with that same route target
value into Spoke VRFs.
With this setup, only the Data Hub VRF accepts routes from the Spoke VRFs, and only the Spoke VRFs accept routes from
the Data Hub VRF. Using this method, the Spoke sites need very little routing information (perhaps just a default route) as
they only need reachability to the Hub site, thereby keeping routing tables small and churn-free.
Figure 4 Network Routing for Single Tenant
The Hub and Spoke example above serve as a good foundation, since the Contrail SD-WAN solution implements route
distribution and separation in the same way when forwarding traffic from one site to another, or when breaking out traffic
to the local internet.
The drawing below shows a Spoke site example configured with two overlay tunnels and local breakout, with all traffic
flowing out the same interface. Each traffic path has its own VRF, and route targets are assigned appropriately at the
Spoke and Data Hub sites to ensure proper tenant route separation.

Figure 5 Local Breakout for Cloud Apps and Direct Internet Access
L3VPN contructs foster the creation of overlay logical topologies to group and connect sites within a tenant. Multiple topologies can
be created within a tenant by enabling network segmentation when creating a tenant. Departments can then be created by function
or location and assigned to sites as needed. Mesh tags can be deployed to build geographical or regional mesh topologies. This
allows administrators multiple levels to segment and control traffic by if they choose.
CSO Overlay (Tunnels) Network

The overlay network includes the logical tunnel connectivity between devices in the SD-WAN environment. Overlay
Tunnels provide a secure point-to-point network independent of the service provider and allows the use of private IP
addressing, route advertisement and ASNs independent of the underlaying networks. Using overlay networks, multiple
layers of network abstraction can be created to run virtualized network layers, which are supported by a physical
infrastructure. The aim of an overlay network is to enable a new service or function without having to reconfigure the
entire network design.
Figure 6 Cloud Delivered Branch
The Diagram above shows an overlay network for a Hub and Spoke environment. Each Spoke has two tunnels to carry
traffic to the Hub: one through the private MPLS cloud and one over the internet. Another option would be to have one
link broadband and a backup LTE link. The OAM traffic will be carried over an IPsec tunnel via designated WAN interface.

CSO Overlay tunnels can be either MPLSoGRE or MPLSoGREoIPSec. CSO automatically provisions and establishes these tunnels as
part of the deployment process independent of the underlaying infrastructure.
Operations and Maintenance (OAM)

Secure OAM was introduced to provide a communications channel between CSO and SD-WAN devices that is independent of
local WAN interface. WAN IP addresses may change as in the case of Spokes that obtain addresses from ISP DHCP. Secure-OAM
enables SSH sessions, BGP traffic and syslog information over secured and redundant IPSec tunnels that terminate on Provider
Hubs. SD-WAN managed Spoke and Hub devices all use OAM tunnels for services.
All Hub and Spokes use the device loopback address for OAM (auto generated by IPAM internal to CSO), for provisioning
and maintenance. As shown in Figure 6, the initial configuration that is pushed to device will provision the loopback
address and create an IPsec tunnel that terminates on the configured OAM Hub. The second stage of provisioning will
create additional IPsec tunnels terminating on Data or Enterprise Hubs and configure BGP for route import and export. Site
activation will configure Spoke and Hub devices including any required VRFs, routing policy and security policies.
Redirect.juniper.net
1 Device connects to
CSO redirect.juniper.net.
2 Device contacts CSO

2 3
1
3 CSO pushes Stage-1 config with
loopback address to device.
4 device establishes Ipsec tunnel to

4 hub for OAM brings up BGP to learn
Spoke#1 5 routes
5 CSO configures additional Ipsec

tunnels to hub and import /export
policies
Data / provider Hub
S-S MPLS tunnels

S-S Internet tunnels
Site tunnels to Hub/Gateway sites
Traffic between sites
CSO to Device communication
1
Juniper Internal
Figure 7 ZTP Provisioning Process
CSOaaS will provide the loopback address for Hubs and Spokes. On-Premise installations have the option of choosing a
loopback address space that will be added to an internal CSO IPAM. Once CSO is provisioned it will automatically allocate
from the internal pool of OP address unless an administrator overrides the default addresses by providing an loopback
address during site creation.

Let’s look at some other ways you can optimize topology within a tenant.
Mesh Tags
Mesh tags are arbitrary labels that enable an administrator to create additional sub-topologies within a tenant. The default
tag names are Internet and MPLS, but any tag name may be used. The only requirement is that Spokes and Enterprise Hubs
must share like tags if you want the site to be able to peer to a given gateway site. This creates a lot of interesting ways to
segment traffic, increase redundancy and scalability.
These rules apply to mesh tags:
• Spoke sites support one mesh tag per link.

• Clustered spokes support four mesh tags per site, otherwise it is three mesh tags per site.
• Enterprise Hubs support multiple mesh tags per link.
• Provider Hubs do not use mesh tags.
• Site-to-Site tunnels must have a matching mesh tag
• Up to three mesh tunnels is supported between sites in a mesh.
Use Cases
Use cases for mesh tagging include:
• Connecting different Underlay Links, Mesh tags allow different link types to be connected as long as they have a
common Mesh Tag.
• Site to Site Tunnels based on capacity – Assigning Mesh Tags by capacity will prevent a high-speed link from over-
running a lower speed link.
• Dual CPE may not always have the same number of WAN links - Mapping Mesh Tags to links allows better traffic
distribution on sites with a higher number of WAN links. In the past those links would have been underutilized.
• Geo-Meshing – Establishing regional Mesh Tags allows a Tenant administrator to group sites by geography and
distribute load on Enterprise Hubs.
• Dynamic Mesh Load Balancing – CSO will load balance on sites with multiple links with the same Mesh Tags.
• Redundant Links – Established by specifying the same Mesh Tag to multiple WAN links. This can be a site with two
WAN links and a Mesh Tag common with a site that only has a WAN single link.
Figure 7 illustrates some of these use cases.

Connecting different underlay links Site-to-site tunnels based on capacity
WAN_0 Overlay tunnel on

WAN_0
WAN_0 Wan links with same
MPLS MPLS wan-links wan_1 on WAN_0
MPLS, 10G MPLS, 1G capacity can be tagged
Tag: T1 Tag: T1 the two sites can be
with same mesh-tags.
created even though Tag: GOLD Tag: SILVER
they are of different
underlay type (MPLS,
WAN_1 WAN_1
MPLS
Internet). WAN_1 WAN_1
Internet
MPLS 1G MPLS, 10G
Tag: T2 Tag: T2
Tag: SILVER Tag: GOLD
Site#1 Site#2
Site#1 Site#2
Geo based meshing With Dual CPE

Gateway Site
WAN_0 WAN_0
Tag: CPE1-MPLS Tag: CPE1-MPLS
Tag: MPLS-USA CPE 1 CPE 1
Tag: MPLS-IND WAN_1
Tag: CPE1-MPLS
WAN_1 WAN_2
Tag: CPE2-MPLS Tag: CPE2-MPLS
CPE 2 CPE 2
WAN_3
Tag: CPE2-MPLS
MPLS-IND MPLS-IND
MPLS-USA MPLS-USA
INET-IND INET-IND Site#1 Site#2
INET-USA INET-USA
Site#3 Site#4
Site#1 Site#2
© 2017 Juniper Networks, Inc. All rights reserved.

1
Figure 8 Sub-topologies created with mesh tags
Use mesh tags to compartmentalize and separate traffic, create sub-topologies, and add additional redundancy. Use
descriptive names that allow the viewer to understand what components will be meshed together, i.e., NA-West,
Accounting etc.
Controller Based on-demand Mesh Dynamic Virtual Private Network (DVPN)

DVPN allows a tenant to support multiple topology types simultaneously. Spokes configured with security policies allowing
connections to other Spokes can establish sessions with other Spokes dynamically independent of the controller. Logging
information from every traffic flow is forwarded to CSO which will track closed sessions and when the administratively set
trigger point is met CSO will attempt to provision a DVPN tunnel between the Spokes. Administrators can set the trigger
threshold at tenant and Spoke level. If a different threshold is set for the spoke, then it will override the threshold set for
the tenant.
DVPN can only be triggered for Spoke to Spoke traffic where the Spokes have routable IP addresses or via NAT Hole-
Punching. In the case of NAT Hole-Punching, CSO incorporates STUN server functionality and real-time monitoring to
decide when to push configuration to establish D-VPN tunnels.
Figure 8 illustrates the DVPN establishment sequence.

Figure 9 Dynamic VPN Establishment
CSO monitors the sessions and will terminate tunnels based on tenant specified Key Performance Indicators (KPI). The
default threshold value for creating DVPN tunnels is 5 (sessions closed measured over a two-minute period). The default
threshold for deleting DVPN tunnels is 2 (measured over a 15-minute period). Administrators can modify default
thresholds at global, tenant or site level.
DVPN tunnels should be used to reduce hair-pinning of traffic through Hubs and to off-load Hubs when possible. For sites
that will communicate frequently the KPI for DVPN default value for establishment can be left at default while the session
teardown should be changed to 5. This will reduce the frequency of configuration changes by enabling long-lived DVPN
tunnels and reduce tunnel churn.
Local Breakout Options -FIX

Traffic may be broken out to the underlay networks as desired rather than terminated at Provider or Enterprise Hubs. You
can specify which breakout option to use when specifying Service Level Agreement (SLA) Intents. SD-WAN policy intents
help optimize utilization of the WAN links and efficient distribution of traffic. SD-WAN policy intents are applied to source
endpoints (such as Spokes and departments) and destination endpoints (applications or application groups) and can be
defined for Spoke-to-Spoke traffic (by using SLA profiles) or for breakout traffic (by using breakout profiles) for cacheable
applications. Application caching is the mapping between application type and destination IP address, port, protocol and
service. Non-cacheable applications (such as YOUTUBE) will follow site /department specific default behavior configured
by the user.
Local Breakout is analogous to split-tunnel terminology on the same interface where some traffic will go directly out the underlay
network versus all traffic going out the overlay or VPN network. When enabled the devices routing table is updated to have default
route pointed at a given interface and then longer prefix matches will go through appropriate overlay tunnels. This allows the site
administrator to create SLA policies which determine when traffic should egress from overlay network to take alternative routes to
a destination point. Enterprise Hubs should have a separate interface for underlay breakout to increase traffic visibility on the
Enterprise Hub.

Local Breakout options:

• Local Breakout (LBO) – By default LBO is disabled but can enabled when creating a Spoke. CSO will then add a
default route to LAN-Department(s) to forward unknown prefixes to the next-hop address of the egress interface.
This impacts all applications going out to SaaS or Internet. No selective breakout based on application can be done
in this model. However, user can configure APBR policy to select the underlay interface type (MPLS/Internet) that
will be used for the breakout. This can be configured on a per application basis. Local breakout can be configured
by Spoke site, or by department if network segmentation (enable l3vpn per department) is selected when creating
the tenant.
• Central Breakout (CBO) – is a breakout that can be shared by multiple spokes at a centralized point. For a spoke, if no local
break is configured, central break out can happen at the Enterprise Hub once a Firewall intent with Central breakout policy
is applied, traffic will be dropped until the policy is applied. CBO is the default type when no other breakout is configured.
• Cloud –SLA Policy selected traffic can be forwarded to a Zscaler instance in the cloud. An IPSec tunnel is established from
CPE to Zscaler and then Zscaler can perform security functions in addition to what CPE provides.
For Underlay breakout types the user will be able to configure intent-based breakout policies for site or department
without specifying a specific application (application ‘any’) so that the rule applies to ‘all’ applications. LAN type breakout is
purely routing based.
Breakout Profile and Breakout Policy Management

The Breakout Profile contain the preferred link type to be used for the profile, the profile name, and breakout profile type.
Breakout Profiles are a CSO construct that overrides the Default Route supplied by the enabling local breakout during site
configuration. This breakout profile is very granular and can be applied per Spoke, or per Dept, or per Dept per Spoke, and
can be application aware. The exit point for traffic is always the underlay interface. If it is programmed to be application
aware the traffic matching the application uses the profile while the unmatched traffic uses the breakout from one of the
Spoke LAN Breakout, or the other Enterprise Hub site LAN breakouts.
The administrator can configure intent policy rules with the created breakout profiles with site/site-group/department and
application as target. The Breakout Profile has the following types:
• Local Breakout (Underlay): Select this option to break out traffic locally (on the underlay) from the site.
• Backhaul: Select this option to break out traffic through a Provider Hub or Enterprise Hub (if configured).
• Local Breakout (Cloud): Select this option to break out traffic through a cloud-based security platform (Zscaler).
In addition, the administrator can specify:
• Name – Name of the Breakout Profile that will be used in the SD-WAN Policy.
• Description – Description for the Breakout Profile.
• Traffic Type Profile - Select a traffic type profile to apply class of service parameters to the breakout traffic. You can
select only a traffic type profile that is enabled.
• Preferred Path – Select the preferred path to be used for breaking out the traffic (MPLS/Internet/ANY).
• Rate Limiting: Select to enable rate limiting of breakout traffic for cacheable applications. (Optional).

Customer corporate policy will dictate which breakout policy (if any) will be deployed. Some customers will want to
have internet bound traffic egress their links asap while others may require that all traffic passes through an inspection
point. The breakout options provide flexibility to allow customers to hand-off or process traffic at different points in the
network based on corporate policy requirements. In general, Breakout policy allows Internet bound traffic to egress at
first possible opportunity to offload traffic to the Internet while optimizing IPSec capacity.
APBR and SLA Management

Advanced policy-based routing (APBR) enables you to define routing behavior and path selection per cacheable application
(group). The APBR mechanism classifies sessions based on well-known applications and uses policy intents to identify the
best possible route for the application. Dynamic application-based routing makes it possible to define policies that will
switch WAN links on the fly based on the application's defined SLA parameters. Understand that it requires for CPE to
have multiple WAN uplinks with defined as type internet or type MPLS.
Real-Time Optimized, also known as Application Quality of Experience (AppQoE), a data plane-level mechanism that
provides better scalability and faster decision making. Also, working in conjunction with APBR, AppQoE functions at the
device level, that is, the devices themselves perform SLA measurements across the available WAN links, and then
dynamically map the application traffic to the path that best serves the application’s SLA requirement. Unlike bandwidth
optimized mode, this is all done without the need of the CSO controller to distribute SLA-specific routes.
With AppQoE, when an SLA violation occurs, only traffic corresponding to the application that reported the SLA violation
is moved to an alternate link; any other traffic using the link is unaffected. Link switching is done at the application level by
the Spoke device. Only traffic related to an application that is in violation of SLA policy is moved to another link if available,
other applications will remain on the link unless they report an SLA violation.
Service Level Agreement (SLA) Policy and Profiles.

SDWAN Policy intents help in optimum distribution of WAN Link utilization and load distribution. They are applied to
source endpoints by applying associated SLA Profiles to selected sites, site groups, or departments. SDWAN supports
Advanced Policy-Based Routing to define routing behavior based on applications and to switch links dynamically based on
SLA Profiles.
SLA Profiles are created for applications or groups of applications for all tenants. The consist of configurable constraints
such as path preference, SLA Parameters including throughput, latency, jitter and loss. You can also define Class of Service
(COS) and rate limiters for upstream and downstream traffic. They can be used to map applications to breakout options or
map a given source (Spoke site, department, group) and selected applications to an SLA profile. There are predefined
profiles supplied with CSO for the most common use cases as shown in Table 3. You can also define and create custom
traffic types. The traffic types map QOS to Forwarding class and scheduler queues and packets are marked with
appropriate DSCP values. It is also used to map type of probe for that link / traffic type.
Voice and video tend to have the same loss, jitter and latency considerations. Typically, loss must be <1%, jitter <30ms,
latency <150ms. You can add profiles for individual traffic types based on this and fine tune the rate limiting per
application. Data applications may be able to withstand higher packet loss and jitter depending on the application but
machine to machine application performance will need voice/video type SLA profiles.

BW with
Loss Jitter Latency Ethernet
Encapsulation
Voice – G711 <1% 30ms 150ms 93Kbps
Voice – G729q <1% 30ms 150ms 37kbps
Video Conference <1% 30ms 150ms 395Kbps
Video Streaming <2% N/A 5 seconds Variable
Skype Video <1% 30ms 100ms 500kbps
Interactive <1kbps
Transactional 0 2-3 seconds 1kbps
Best Effort Web Variable Variable Variable Variable
Table 1 SLA considerations for Branch traffic profiles
AppQoE provides the fast link failover for traffic although it does require the Spoke to have more than one WAN
interface. If LTE is used as a failover link, then you should choose which applications will failover to LTE link with an SLA
profile as it might not have enough bandwidth for all applications.
Default profiles can be used to create SDWAN policies or they can be defined at the global level. When defining a
profile group like applications,
1 configure
© 2018 parameters
Juniper Networks and decide
J3.02.P01.T07 Rev 17 the order of failover.
Template Owner: Radhika Narayanan © 2017 Juniper Networks,
© 2016 Ju
Another option is to use bandwidth optimized in conjunction with Link Cost so that traffic will always use the lowest
cost link that meets SLA requirements.
Configuration Templates
Customers often have base configurations that are applied to all deployed devices, often referred to as golden configs.
These configurations may contain settings to configure syslog, RADIUS, SNMP or many other settings which remain
constant across all deployed devices within the customer environment. Other settings such as routing protocols or routing
policy will be unique per device. For example, a devices BGP configuration will often be unique per device. Configuration
templates enable network administrators to push common configurations to all sites within a tenant or settings that are
unique per device.
CSO Configuration templates can be configured at Global, Opco or Tenant levels to allow the re-use of templates, as well
as Tenant isolation, keeping the Configuration Templates private. Many common templates have been created by Juniper
engineering and have been pre-assigned to device templates to facilitate out-of-the-box and tested configurations. Pre-
assigned templates can be removed from device templates within the tenant view or new templates can be assigned to
tenants by tenant admins. If unique Configuration templates are required, CSO 5.1 includes a template workflow to create
Configuration templates by importing CLI statements and then creating and defining variables to build the final template
this allows any valid Junos configuration to be templatized.

Configuration Templates inherit initial parameters or default configuration values (if given), or the admin can choose to add
unique values by tenant or by site. Template configurations are unique per level of granularity. For instance; a
Configuration template applied to a site may have unique values per that site and override configuration templates at
tenant level.
Configuration templates can be deployed at the end of device provisioning or anytime as long as device is managed by
CSO. The templates can be applying or do groups of devices within a tenant.

Device Types
Devices are defined as Spokes or Hubs; the different functionality is enabled by Device Templates associated with the site.
Spokes are usually located at branch locations and based on the device template may be NGFW or SD-WAN. Hubs are
always SD-WAN as NGFW does not use Hubs. SD-WAN requires an OAM capable Hub for control traffic, and a Provider
or Enterprise Hub as an anchor point to terminate overlay tunnels. Spokes may also establish tunnels to other spokes
based on traffic flows.
Single or Clustered SRX or NFX

SRX and NFX devices can be deployed as standalone or in a cluster. The cluster provide the benefit of stateful fail-over in
the event one of the routing-engines has an issue. Links are placed in RETH (redundant ethernet) interfaces and then in
redundancy groups. The CSO cluster template will place WAN links in their own RETH interface and then the LAN links
will share a RETH interface. Each of the WAN links will require an IP address but the LAN Links will share one or more IP
addresses.
Cluster deployments may terminate WAN connections on a switch and then feed the same connections to multiple links on
the cluster as shown in the diagrams below.
Advantage of Standalone SRX/NFX
• Device independence – best fit
• Directly connect WAN links
• Easy upgrade
• 100 percent IPSec performance
• Add devices as needed
Disadvantages of Standalone SRX/NFX
• No control plane failover
• No stateful persistence
• Manually build Lag
• VRRP requires Configuration Template
Advantages of Clustering SRX/NFX

• Geo-Redundancy
• Synchronized Configuration
• Session Maintenance
• Local Fail-over and option for Multi-homing failover
• RETH interfaces for LAN connections
Disadvantages
• Requires twice as many devices for equivalent IPSec capacity
• Same device model required for both devices
• No interface redundancy in CSO configuration
• Stateful failover only if WAN links go down
• Upgrades require rebooting both devices simultaneously
• Tunnels are tied to Interface – no failover

Hub Devices
Hubs are used to terminate OAM (control) and data overlay tunnels from Spokes. CSO SaaS will provide the OAM
functionality while On-Premise deployments must configure OAM functionality on at least one Hub. All Hubs must have
public IP addresses or routable IP addresses in a private network. On-Premise HUB deployments may be defined as OAM,
OAM and Data, or Data only.
CSO SaaS Characteristics

• OAM functionality is provided by the service
• Provider Hubs are created at Opco level and imported per tenant within that Opco.
• Provider Hubs can be shared among tenants within the Opco
• Enterprise Hubs must be deployed at tenant level.
CSO On-Premise Characteristics
• One Hub must be defined as OAM or OAM/Data and can be shared among tenants as a Provider Hub.
• Hubs deployed at global level can be shared among tenants as a Provider Hub.
• Enterprise Hubs may be deployed at tenant level.
Multihoming Hubs
Sites can connect to two of the same type hubs in a process known as multihoming. One hub is primary and the other is
secondary. Sites will always have a route preference for the Primary hub. If the Primary hub is not available, then traffic will
automatically failover and then fail-back if primary site is available again. This behavior allows administrator to select which
hubs are primary and secondary on a site by site basis enabling manual loaf balancing of traffic across hubs. Since there is
potential for overload links in a failover scenario, one should be careful not to overload the link and to prioritize traffic
accordingly.
Provider Hubs
Provider Hubs may be a physical or virtual SRX that is configured to provide any combination of OAM or Data services for
SDWAN Spoke devices. Provider hubs are onboarded at Global or Opco level depending on whether CSOaaS or On-Prem
solution. Tenants can import the Provider hub and when a spoke is on-boarded it will correctly pick the correct devices for
control and data path traffic. Provider Hubs can be shared among multiple Opcos or defined at Opco layer to be shared
among tenants. Each tenant has a virtual slice of the Provider Hub similar to logical systems with independent routing,
security and other network characteristics. Provider Hubs serve as the termination point for overlay tunnels from the
Spokes if the Enterprise hub isn’t configured or in the event that an Enterprise Hub associated with the site is not available.
Provider Hubs can be configured as Primary or Secondary for multihomed Spokes. Provider are not designed to be used as
Gateways to Datacenters or remote campuses, for that we use Enterprise Hubs.
When onboarding a provider hub:
• Onboard at Opco or Global level (on-prem only)
• No ZTP – Stage-1 is pasted in but that allows you to verify network connectivity and fast fail.
• Configure NAT for egress traffic
• Provider Hubs put WAN interfaces in different routing tables so you may need to add a default route for inet.0
interface.
• Additional configuration is required for L2/L3 VPN connections or dynamic routing with legacy network.
Enterprise Hubs
Enterprise Hubs are an extension of the SD-WAN CPE template that are created and used by Tenants and may be
considered somewhat of a self-managed Hub. Enterprise Hubs are not shared among tenants. They serve as the primary
termination point for Spoke overlay tunnels. When a Spoke is associated with an Enterprise Hub than the Enterprise Hub
will take on the role of the Provider Hub with regards to data traffic. In that case the Enterprise Hub will advertise a default

route to its associated Spokes. In the event that an Enterprise Hub fails then Spoke traffic will failover to the associated
Provider Hub (On-Premise version only).
Enterprise Hubs do not replace OAM capable Hubs and there must always be at least one OAM-capable Hub. Enterprise Hubs
must have a public IP address or a routable address in a private network. Enterprise Hubs support OSPF and BGP neighboring with
customer routers or switches. They can learn customer prefixes and advertise those prefixes to other Hubs. They will advertise a
default route to attached Spokes and can be used for CIBO if breakout policy is applied.
Both the Provider Hubs and Enterprise Hubs may serve as an anchor point for Spokes to communicate with other Spokes.
Spokes that are behind Nat have to use a Hub and Spoke topology where IPSec termination point is either a Provider or
Enterprise Hub. If Spokes have routable addresses, then DVPN tunnels can be triggered for Spokes to communicate
directly with each other. Both Provider and Enterprise Hubs can provide NAT services for outbound traffic.
You can have multiple Enterprise Hubs per Tenant, in fact, you can add 5 or more. When you have multiple Enterprise
Hubs within a Tenant they will auto-mesh using BGP with a GRE Tunnel/IPSec control channel between Enterprise Hubs.
When you create an Enterprise Hub, you specify Mesh Tags to define which Spokes can create DVPN tunnels to the
Enterprise Hubs, you can use Site edit to add or change Mesh Tags in the CSO 5.2 or greater release.
Enterprise Hubs are larger SRX devices in Juniper’s Contrail SD-WAN solution and serve multiple purposes such as:
1) Exclusive per tenant use

2) Hub functionality to connect different Spokes together
3) Datacenter eBGP or OSPF neighboring to learn local routes and interconnect to legacy enterprise network.
4) Automatic Dynamic mesh with other Enterprise Hubs to advertise their Spoke subnets to other Spokes creating a
fully connected SD-WAN network
5) Centralized breakout option.
With the above functionality one can understand why a larger SRX is required to achieve the proper scale of an SD-WAN
network. As with building physical networks, an SD-WAN network must have a governing set of rules to create
predictability, availability and reliability. These rules are not set in stone and are more guiding principles with Solution
Validation testing, by Juniper Networks to give you the correct direction on this SD-WAN journey.
CSOaaS – OAM hubs are provided by the solution. Provider Hubs or Enterprise Hubs are required for data path
termination.
On-Premise Solution - At least one Provider Hub with OAM capabilities must be deployed. Juniper recommends
deploying at least two so there is OAM redundancy.
In either solution locate the Provider hubs in the natural direction of traffic to optimize traffic flows. Import multiple
instances to tenants so the Spokes have a failover option.
Enterprise Hubs anchor associated Spokes and should be placed in strategic locations within the network such as,
Datacenters, headquarters, Network Aggregation points, etc. When multiple Enterprise Hubs are deployed within a
tenant, they create a Dynamic mesh between each other, and we must consider the (N squared -1) scaling problems
that come from a Dynamic mesh of anything. The Enterprise Hub full meshing is automatic with the Contrail SD-WAN
solution to create ‘All Sites’ connectivity, although there is no restriction on how many Enterprise Hubs a Tenant may
create so be aware of the dynamic meshing and number of tunnels created.

Spoke Devices
The CPE device at an Enterprise customer’s branch site acts as a Spoke device in the SD-WAN model. The Spoke is the branch
router providing connectivity from the branch site to other sites in the tenant network, to cloud services, and to the Internet. Spokes
can also provide SD-WAN, UTM and SSL–Proxy Security Services, when licensed and configured in addition to standard branch
router services such as DHCP, Firewall and NAT. CPE can be physical or virtual device located on premise or cloud based.
On-premise Spoke
As of CSO Release 5.1.2, the supported Spoke devices and their interface types are listed in Table 3.
Table 2 Supported WAN interface types
SRX Series Security Devices

The SRX security device Spokes combine security, switching, routing and WAN interfaces for small to medium size
distributed environments. Different variations of the SRX family are certified as CPE with CSO. They are SRX-
300/320/340/345/550m/1500/4100 and 4200. SRX devices can be onboarded via Zero-Touch-Provisioning or by
minimal touch where the stage-1 configuration is pasted into the target device and then it will be provisioned by CSO.
NFX Network Services Platform

The NFX platform differentiates from traditional CPE devices in that it can host multi-vendor Virtual Network Functions
(VNFs) and support service chaining, managed by Juniper Networks’ Contrail Service Orchestration. NFX devices eliminate
the operational complexities of deploying different physical CPE devices at a customer site as it combines physical and
virtual entities in a single piece of hardware.
NFX devices provide the ability to spin up virtual functions on demand with a key VNF supported on the NFX platform
being the Juniper Networks’ vSRX. With the vSRX residing on the NFX all the feature-rich security services found on a
standard SRX Series Services are available. The NFX Series comes in the 150 and 250 models of which the NFX150s can
be provisioned over LTE while both the NFX150 and NFX250 can use the LTE as a backup link with the NFX 250 utilizing
an USB dongle for LTE.

vSRX as a Spoke or Hub

vSRX on a x86 or whitebox can provide Spoke or Hub service depending on configuration. You can adjust the CPU and
memory allocation to increase throughput and performance as determined from a network survey and proper planning.
The target vSRX must be brought up manually or via external means and one of the vSRX interface IP address has to be
reachable for CSO to on-board and finish configuring the device. Recommend using vSRX3.0 for performance reasons.
Cloud Spoke
Cloud Spokes are devices that serve as gateways to a cloud provider environment, seamlessly connecting the Virtual
Private Cloud into the SD-WAN environment. SD-WAN Spoke devices can be located in an AWS VPC natively, but on
other public clouds it is treated as a White Box implementation. When a vSRX instance in the VPC is provisioned it will
serve as the cloud Spoke device with the appearance of being another Spoke end point on the SD-WAN network. Cloud
Spokes can also be provisioned as Enterprise Hubs.
Device Deployment Overview.

Juniper SRX, NFX and vSRX can be automatically provisioned through a process call Zero-Touch Provisioning. A device with
factory default configuration with internet access will connect to Junipers Redirect service and then connect to CSO for
provisioning. CSOaaS will automatically populate the redirect server when a site is configured. On-premise deployments can access
the Redirect Service, but serial numbers have to be manually added. The SD-WAN ZTP process is shown below.
Redirect.juniper.net
1 Device connects to
CSO redirect.juniper.net.
2 Device contacts CSO

2 3
1
3 CSO pushes Stage-1 config with
loopback address to device.
4 device establishes Ipsec tunnel to

4 hub for OAM brings up BGP to learn
Spoke#1 5 routes
5 CSO configures additional Ipsec

tunnels to hub and import /export
policies
Data / provider Hub
S-S MPLS tunnels

S-S Internet tunnels
Site tunnels to Hub/Gateway sites
Traffic between sites
CSO to Device communication
1
Juniper Internal
Figure 10 ZTP Provisioning Process
Stage-1 configurations are based on device deployment templates that contain configuration and provisioning settings for a physical
device, such as a Spoke device or a Hub, which you manage through Contrail Service Orchestration (CSO). The CSO installation

includes several default device templates for standalone and clustered devices which can be deployed as presented or customized
as required. Each Device template is specific to the type of device and type of deployment. You must assign a device template to
each Spoke device at the site.
The two types of information included in the device templates:
Basic information—It prepares the device for remote activation with a base configuration to configure interfaces, default
routes, and connectivity to OAM, Enterprise, or Provider Hubs as required by device type and site configuration
parameters. The number of links and type of links will also be specified.
Configuration template information (requires admin access to create and assign to tenant)—It specifies the additional
settings that can be configure for the device. For example, you can enable configuration of LAN and firewall policies. You
create these configuration templates using Configuration Designer, or import existing templates, and then optionally add
settings per device.
Configuration templates are used to customize settings particular to an organization. Common types of configuration templates
include system settings such as logging, NTP and DNS, or protocol configuration including VRRP, IGMP or LACP. The templates are
provided by Juniper Engineering and included with both On-Premise and SaaS solutions. In addition, tenants can create their own
Configuration templates via CSO Configuration Designer and added to a device template.
Standardize what ports will used for WAN and LAN interfaces. If more than one template is required clone a template, make the
changes and test. If Spokes require Configuration templates, copy the Junos CLI snippet into Configuration Designer, create and
save the template. Associate it with a cloned template and test. Use variables if Spokes will have different parameters for the same
cli stanza and set the Configuration template to set auto-deploy to no. Then after provisioning is complete select the device in GUI,
fill in the fields and deploy.
Active/Active or Active/Backup
LTE
Definition Dynamic
LTE
MPLS
BB
BB
Policy
MPLS
LTE
Prescriptive
BB
BB
Active / Backup Active / Active
WAN Link Utilization

© 2018 Juniper Networks J3.02.P01.T07 Rev 17 Template Owner: Radhika Narayanan © 2017 Juniper Networks, Inc. All rights reserved.
Figure 11 Branch site requirements

During Spoke site configuration, you can designate WAN links can be active/ active or active / backup, independent of link type as
shown in Figure 7. How the link fails over to the backup or how the link load-balances across multiple active links will be used to
configure the Spoke site.
Prescriptive links are a static configuration. Traffic failover will occur when routing updates are received. A couple examples are
below to give the reader an idea of the traffic steering methodology.
• Voice over MPLS and Everything else over broadband
• In the event of a link failure only Point-of-Sale goes over LTE as it is a metered link.
Dynamic links use AppQOE to test the quality of the link and can make local traffic forwarding decisions based on loss, delay and
jitter thresholds.
Single Active Prescriptive – Residential or retail Spokes where Internet access is non-essential for business continuity. Traffic only
fails over to secondary link via intervention from routing update. Simplest and most common deployment for retail Spokes where
multiple active wired links are not cost effective.
Single Active Dynamic - Retail or business with POS terminals that requires Internet access and fast failover if a link fails. Failover
decision based on link quality and then failback if CPE determines primary link is again valid.
Dual Active Prescriptive – Campus type environments with routing policy to select links. Any change to link quality requires an
upstream change to affect the link. This is the case if RPM was used and the link degrades. Standard deployment for years.
Dual Active Dynamic – Banking or some businesses that requires Internet access uptime. SLA for active link measurement and then
decisions local to CPE to determine which applications will be on which links. Fast failover, link optimization, can separate traffic into
bundles and control failover / fallback.
Understand your site requirements, available links per site, and tolerance for latency and jitter. LTE should be used as
backup to minimize usage-based charges. Build an SLA to offload internet bound traffic and to prefer broadband links over
cellular access.
Scaling
There are a number of considerations that impact scaling including the number and types of devices that will be deployed. SDWAN
vs. NGFW deployments typically effect sizing requirement more than the number of devices deployed as we need to consider Ipsec,
BGP and Meshing. In general about 3500 sites are supported by HA Production server but mileage varies based on the type of sites.
Please refer to the CSO Installation and Upgrade Guide for further details.
Factors that Affect Scale

Based on the topology and device types we can start looking at how many devices can be used in different scenarios.
There are a number of factors which will influence the number of Spokes a Data or Enterprise Hub can support.
• BGP neighbors scale limit

• GRE tunnel scale limit
• IPsec tunnel scale limit
o Including both data and OAM
• VRF scale limit with Throughput-optimized SD-WAN mode (RPM)
o Each tenant results in around 11 VRFs on associated Hub
• Impact of Realtime-optimized SD-WAN mode (AppQoE) on throughput

o Especially for Hubs

• Platform Throughput limit with GRE, IPSec and MPLS encapsulations
o Especially for Hubs
• Events per second processing limit of CSO deployment environment
• CSO qualified/supported scale of on-demand site-to-site VPNs
BGP Sessions Supported

The following table depicts BGP sessions between devices. For instance, a Spoke will have a BGP session for OAM and one to VRR.
The Spoke OAM session may either be on a dedicated OAM device or two the associated Provider Hub with OAM capabilities. If
the Spoke is multi-homed it will have BGP sessions to both Provider hubs. The Spoke will not have a BGP session to associated
Enterprise Hub. It will learn the routes from the VRR.
Figure 12 BGP Control and Overlay Tunnels.
SRXs support up to 1000 BGP peers per device so that should not be the gating factor. VSRX are qualified at 200 BGP
peers but should scale higher given enough RAM and CPU. Table 6 is derived from the scaling sheets and based on the
number of VRFs or GRE/IPSec tunnels supported, we can determine they won’t be a gating factor either. It turns out the
average throughput we want a site to have is the gating factor.

Table 3 SRX scaling for SD-WAN deployments
IPsec performance
During site creation the administrator can specify which links will carry OAM traffic and if WAN links will establish either GRE or
Ipsec tunnels to Provider / Enterprise Hubs for data termination and enable D-VPN tunnels to other Spokes The number of Spokes
that a Hub can support is dependent on the number of tunnels and the IPSec throughput capabilities supported on that particular
platform. The throughput factor can be somewhat mitigated by configuring Local Break Out so that traffic will route directly to the
Internet (AKA: Underlay) vs. over the tunnel connections to other Spoke or Hubs. The IPSec scaling numbers can be found at:
https://www.juniper.net/us/en/local/pdf/datasheets/1000265-en.pdf
Spokes with IPSec tunnels on one or both WAN interfaces will have those plus the OAM overlay tunnel terminating on a Provider
Hub or separate OAM Hubs. If a Spoke is multi-homed to two Hubs, then both Hubs may have OAM and DATA tunnels for a given
Spoke, supplying high availability.
It is important to look at the applications required by a tenant and associated sites when configuring the end locations. Software as a
Service (SaaS) applications can be directly accessed and usually don’t require traffic to originate from a centralized location. In that
case, you should specify LBO and offload the traffic from the IPSec tunnels transiting the Hub, which could tremendously increase
the scale of the SD-WAN deployment. If LBO is configured, then only VPN traffic, monitoring logs, and AppQOE (if enabled) will
utilize Provider Hub bandwidth allowing more Spokes per Provider Hub. The following example table has a list of common
applications with and without IPSec tunnel overhead. Multiplying the number of users per Spoke would give approximate
bandwidth requirements for local breakout and Provider or Enterprise Hub IPSec processing requirements. To get a better estimate
of site requirements copy the numbers into a spreadsheet and add a column for the number of users.
Traffic Estimate LBO IPSEC Tunnel w/overhead

Email - Cloud hosted 1.15 1.15 1.22
Email Corporate hosted 1.15 1.15 1.40
Backup 2.00 2.00 2.11
Cloud Services 1.50 1.50 1.59
File Sharing 0.50 0.50 0.53
Messaging 0.50 0.50 0.53
Web Browsing 0.33 0.33 0.35
Video Conferencing 4.00 4.00 4.23
VOIP calls 0.01 0.01 0.01
VoIP video Calls 1.28 1.28 1.35
Wi-Fi 1.00 1.00 1.06
Point of Sale Terminal 1.50 1.50 1.59
Total Bandwidth 14.92 15.96

Table 4 Application bandwidth requirements
Given the qualified numbers from the scaling sheets we recommend the following number of Spokes per Hub based on
average IPsec throughput. If sites require additional throughput you can adjust the number of spokes accordingly.
Table 5 Spokes-per-hub recommendations based on throughput requirements
DVPN
We also need to consider the number of DVPN connections supported. As we can see from the following table, there are
sufficient DVPN tunnels available for most deployments.CHECK
Description Scale
Number of full-mesh DVPN tunnels supported per tenant 50000
Number of full-mesh DVPN tunnels supported across a CSO installation 125000
200 tenants with 10 sites per

Number of full-mesh DVPN tunnels qualified across a CSO installation
tenant
Number of full-mesh sites qualified for a given tenant 250 sites
Maximum number of events per second that can be processed be SDWAN log process 90000
Table 6 DVPN scaling details
Additional Bandwidth Requirements

Performance monitoring will also consume some IPsec bandwidth. The active probes used to test links send 1 frame per
session per testing interval. Application tracking logs are forwarded to the analytics engine via logs and will consume 12KB
of syslog per session where the session is any application that is being tracked. AppTrack syslogs are then forwarded via
the OAM IPsec tunnel and will consume part of the available IPsec tunnel capacity.
One last factor to consider is that each Spoke will require the following; one-time and ongoing bandwidth independent of
© 2017 Juniper Networks, Inc. All rights reserved.
VPN traffic. © 2018 Juniper Networks J3.02.P01.T07 Rev 17 Template Owner: Radhika Narayanan
• Stage-1 Configuration: ~20KB

• SRX Image upgrade (optional): 260GB
• Configuration Templates ~ 100 Kilobytes approx.

• Application Tracking and RT Flow – 12KB of syslog per session
CSO 4.1 and after recommends a minimum of 2Mbps per link. After the process is complete there is a minimal amount of
syslog and maintenance traffic on the OAM link, mostly for log files and netconf traffic that is checking for configuration
synchronization.
Characterizing the traffic types and application bandwidth requirements will assist with sizing Hubs appropriately. Spokes
can be categorized as small, medium, and large based on expected IPsec throughput and mapped accordingly. It is better to
add 30 percent to expected capacity requirements and specify a larger Hub with higher throughput than discover there is
contention for IPsec throughput.
Deployment Considerations
This guide builds upon the CSO Deployment and Installation Guides to help you deploy a production ready Contrail Service
Orchestration environment. Follow the instructions given in the Installation Guide to provision CSO as you will find the
server requirements and guidelines for installation. The main focus of this document will be SD-WAN design principles to
choose the right topologies and devices to meet the requirements of the branch offices connecting via Juniper Network’s
Contrail SD-WAN solution. Read the section below to understand when to use public vs private IP addressing and how
that choice may require additional underlay configuration.
Internet access is required during On-Premises CSO installation and also to download signature databases and Junos
images. Ensure that the underlay is configured, and routing is working properly. Hubs of both kinds will need default
routes for interfaces in inet.0. A Virtual IP address is required for User Interface, Virtual Route Reflector, and South-Bound
Load Balancer. The same address can be used for all with the all, for labs we put the address on a vSRX and use Destination
Nat with port matching rules to forward to the correct virtual machines.
CSO can be deployed with private or public IP addresses or a combination of the two. If deployed with public IP addresses then
Juniper recommends limiting devices with public IP addresses to the services that the CPE devices require for provisioning and
operations and found in the Deployment and Installation Guides.
Public vs. Private IP Addresses

The Spokes WAN interfaces can be configured to use static or dynamic addressing and those address types can be public or private
IP addresses. During the SD-WAN ZTP process the Spoke initiates a connection to CSO, which will push IPSec configuration to the
Spoke. The spoke uses its WAN interfaces for IPSec tunnels and will then use the loopback address received from CSO to BGP peer
with associated OAM capable hub. All subsequent OAM traffic originates from and is directed to the loopback address. All
application data traffic is forwarded over the WAN interfaces, and the type of WAN address, Public or Private, will determine the
logical topology and the flow of traffic.
Spokes without public IP addresses or routable IP addresses in a private network are considered to be “Behind NAT” and
will use a Hub and Spoke logical topology and may form a D-VPN tunnel with another Spoke if there is traffic between the
two Spokes. Spoke-to-Spoke communication initially flows through a Provider or Enterprise Hub, then may create a mesh
topology by establishing Spoke-to-Spoke connections with Dynamic-VPN tunnels.
Example Spoke Behind NAT –Spoke A has a private address and the Spoke B has a public. For host behind Spoke A to
communicate with Host B (which is behind Spoke B) then somehow Host A must not only learn the public IP address that
HOST B is behind but must initiate an incoming session via Host B Spoke. In the case of Spokes behind Symmetrical NAT
then the Hub acts as a gateway to connect the Spokes together via FW policies to permit Spoke-to-Spoke traffic.
UDP Hole Punching can be used for Spokes that are not behind symmetrical NAT if the Spoke is attached to two OAM
Hubs. CSO will provision Spokes with a IKE session and UDP port 500 to enable hole-punching through NAT. When CSO
observes traffic between two spokes which are not behind Symmetrical NAT then it will provision a tunnel between the

spokes. This is only use in link types defined as type “Internet”, MPLS links are assumed to be on routable addresses and
can create D-VPN tunnels without UDP Hole-Punching.
HUB HUB
A LTE LTE LTE LTE
Spoke A with Private IP Spoke B with Public or Private IP Spoke A with Public IP Spoke B with Public IP
Figure 13 Sites Behind NAT and Sites with Public IP Addresses
• If both Spokes are behind Symmetrical NAT, the traffic will always traverse via a Hub.
• CSO will not attempt to NAT interfaces with links creates as with MPLS label as it is assumed there is reachability
on MPLS links.
• Tool like Pystun can be sed from a host behind a spoke to confirm NAT type
IP Services
Juniper Internal
All devices in the SDWAN environment require access to common IP services such as NTP, DNS and optionally DHCP. On-Premise
CSO installations will ask for FQDN which is used to create the x.509 certificate that is used for the ZTP process. It. Is difficult to
change the FQDN after installation so please use correct FQDN for the organization and configure DNS before installing the
servers. In a lab environment you can install a DNS/ NTP services on CSO host and use for testing the solution.
Have DNS and NTP tested and validated before beginning any installation.
Underlay (Physical) Network

The underlay network includes the physical connectivity between devices in the SD-WAN environment. The underlay is a
standard routed network and must be functioning before building the overlay. Each Spoke or Hub can have up to four
WAN links. During configuration, CSO identifies the devices’ WAN-facing interfaces as WAN_0 through WAN_3.
• The WAN interfaces can be VLAN tagged or untagged, as per design requirements.
• At least one WAN interfaces must have a valid next-hop, either statically defined or learned via DHCP or
EBGP.
• All on-premise devices’ MPLS network-facing interfaces must be attached to a single Layer 3 VPN
provider.

• Provider and Enterprise hubs require EBGP for hubs to establish tunnels. The BGP Neighbor must have
routes to CSO servers and the gateways of other devices. Use a separate ASN then the one used internally
for CSO.
• The on-premise devices’ Internet-facing interfaces can be attached to different service provider networks.
It is essential that there is a functioning underlay with routing reachability and any firewall or NAT device is provisioned
and validated. The list of required ports is listed in the CSO Installation and Upgrade Guide.
All Hubs or Spokes that are going to be provisioned must be able to ping their default router and be able to reach CSO.
Network connectivity must be validated before any attempt at building overlay tunnels (see next section) or provisioning.
Generally speaking, an OAM/Provider Hub should have two WAN ports, a designated OAM port and one additional port
for Internet access. Enterprise Hubs require at least two WAN ports, and Internet port and one LAN port. Beginning in
CSO Release 5.2 additional WAN Links can be configured if required after provisioning, although existing Spokes will not
recognize WAN Links provisioned on a hub after the spoke is provisioned.
Some deployments will need to import routes to Provider Hubs. This can be done via manual configuration or through
configuration templates.
CSO Requires certain ports to be open in the customers firewalls to allow for device provisioning and monitoring.
Protocol/Ports Purpose IP Addresses
50/51 IP Protocol 50/51 to be enabled for ESP and All

AH
53 For DNS requests Typically, 8.8.8.8 and similar
443 For Hub device to reach out to Juniper https://redirect.juniper.net

Redirect service and to CSO hosted on Public
Cloud. https://cso.juniper.net
8060 (Only for PKI based authentication

PKI Tenants)
500/4500 For IKE/IPSEC connections to Hubs.
Figure 13 Required open Ports
Logging
All CSO provisioned devices will send log files to CSO to enable monitoring and additional provisioning. Logs are encrypted
and transmitted over ports 3514 and 514. Logs contain session close information, tunnel state and other information
relative to the device. Logs are maintained for 30 days.
Provider Hub Location Options

Provider Hubs for data tunnel termination are created at the Opco level and can be shared among the tenants within the
Opco. Enterprise Hubs are created at the tenant level and cannot be shared with other tenants. Both Provider and
Enterprise Hubs can be distributed geographically, or you can use a central Provider Hub and then geographically placed
Enterprise Hubs depending on the Spoke distribution. Ideally the Hubs should be placed in the path of traffic towards the
CSO instance so traffic isn’t hair-pinned and will have a shortest path to a egress point of network. Traffic can be steered
via policy and configuration to use the closest Enterprise Hub location and then traffic can be broken out wherever the site
admin chooses. If a Spoke has both Provider and Enterprise hubs configured it will always prefer the Enterprise hub for
data termination unless the Enterprise hub is un-available.
Enterprise Hub Locations

These should be placed in front of datacenters or large locations where there is a natural destination point for traffic based
on applications or locations of servers. These are often locations where you would need to advertise routes such as
Corporate HQ, Datacenter or network aggregation points. Enterprise Hubs automatically mesh if they share the same
mesh tags and can be deployed as clusters and also for multi-homing sites. Central breakout policies allow for internet
traffic to be forced to a central location for scrutiny before heading to its final destination. Enterprise Hubs will handle the
brunt of the Ipsec traffic so they should be sized appropriately.
Use the following generalized rules when sizing and placing Enterprise Hub sites:
• Size Enterprise Hub Sites based on the Ipsec bandwidth requirements of Spokes and Hub-to-Hub traffic.
• SRX 1500/ 4100/4200 /4600 can be used for Tenant Enterprise Hub Sites
• Use Mesh tags to load balance across WAN Links
• Use Mesh tags to geographically anchor tunnels based on region
• Specify at least one link for LBO with AutoNAT to enable Central Breakout configuration after provisioning
Spoke Site Configuration Options

On-Premises Spokes Sites can be manually configured, imported using JSON, or by using pre-defined site templates. Site
templates are useful when sites share common attributes reducing administrative input to site specific attributes.
WAN Interface Types – OAM and Data

CSO uses device templates to configure Spokes. Among other items the templates specify WAN interfaces and then the admin can
configure if the interface will connect to a Hub, be used for mesh and have OAM enabled on that link. Spokes with multiple WAN
links should enable OAM on multiple links for OAM redundancy. CSO places interfaces configured as OAM in routing-instance

oam-vrf and WAN interfaces in WAN routing instances. In some designs, it makes sense to add another interface / VLAN if you
need an interface that isn't associated with a routing-instance.
Figure 14 Tunnel requirements for SD-WAN
During ZTP The Spoke will use the WAN interface address (underlay) to contact the redirect server and then to contact the CSO
servers. The CSO server will create a unique stage-1 configuration including loopback OAM address and push it to the target
device. After the configuration is committed by the device all further configuration is through the OAM tunnel. For this to work
correctly all the underlay addresses must be reachable via normal routing.
Spoke Configuration Options
Primary Hub Site

One OAM-enabled Provider Hub site is required by all Spoke sites for OAM and optionally data termination. WAN ports can be
configured as type “MPLS” or type “Internet”. These are labels used for CSO to identify what type port is for SLA purposes and not
the actual traffic type on the port. Spoke sites have to match what is configured on their designated hub site or the Spoke
configuration will fail.
Multihoming
Multihoming is the ability of a Spoke to connect to two different Provider Hub devices to provide redundancy. The Hub
devices function as primary and the secondary Hub devices. If there are multiple Spokes in the system, the same Hub
device may act as primary Hub device for one Spoke and secondary Hub device for another Spoke. That is, the selection of
the primary and the secondary Hub devices is only in the context of a Spoke. The Spoke is connected to both the Hub
devices through an underlay network.
Traffic is switched from the primary Hub to the secondary Hub in the following scenarios:

• The primary Hub is down

• The primary Hub is up, but all the overlay tunnels between the Spoke and the primary Hub are down
• The tunnels are up, but the iBGP session between the primary Hub and vRR is down. In this case, the failover
occurs only after the BGP hold-time expires and the default route is withdrawn
Enterprise Hubs
The Spoke can connect to multiple Enterprise Hubs which can be used for Centralized Breakout or as IPSec termination point.
Select the enterprise Hub in the path or at the location where you expect most traffic to flow towards. Mesh tags are associated
with Enterprise Hubs and not Provider Hubs. Mesh tags have to be created before provisioning Enterprise Hubs or the default tags
of “MPLS” or Internet” can be used and associated with matching tags on the target device.
IP Prefix (for Loopback IP Address)

This is the address that will be assigned to loopback interface for OAM connectivity. All Spokes and Hubs in an SDWAN
deployment will be managed via the loopback address. Loopbacks are automatically generated, during site provisioning. CSO will
create a stage-1 configuration that creates the tunnel interface using the loopback address. The loopback address is persistent and
independent of WAN interfaces that have DHCP addressing. Additionally, loopback addresses are internal to the SDWAN network
and are not advertised externally.
Local Breakout
There is also the option to setup Local Breakout to enable unknown prefixes to egress to the internet. The spoke will learn SDWAN
associated prefixes from the VRR and traffic for known prefixes will traverse the overlay tunnels. Local breakout will have a default
route on selected port and egress to internet at selected WAN interfaces. When you create LBO on a Spoke you may also need to
enable auto-NAT to translate between LAN private IP addresses and WAN interface.
If a Spoke is configured without LBO then all traffic will terminate at the Provider Hub or Enterprise Hub and source NAT should be
applied on the Hub using egress default WAN link as the outbound NAT interface.
Other Options
Spokes with multiple WAN links have additional topology options available:
• Enable Local Breakout

o Breakout Options – Enable Local Breakout
o Auto Create NAT Rule – automatically provision NAT rules
o NAT Translation -Interface or Pool
o Preferred Breakout Link
o BGP Underlay Options
§ BGP neighbors, ASNs, Authentication and Advertise Local Prefixes
• Use for FullMesh- if so, select mesh tag
• Use for OAM Traffic – Link with OAM tunnel, recommend two
• Backup Link – Used when Primary links are not available. Cannot be LBO only link -CSO will monitor primary and
switch traffic back to the primary link when it is back online
• Default Link – allows selection of default link for routing overlay and LBO traffic in the absence of SLA intents.
Multiple links can be designated as default if desired with Equal-Cost Multipath (ECMP). If no default links are
selected, then all WAN links use ECMP.
• Backup Links – CSO will monitor primary / secondary links and move traffic from one link to another when SLA
requirements are not met. LBO traffic will follow backup link state for primary and secondary paths.

There are a multitude of ways to configure a Spoke with regards to expected traffic patterns, redundancy requirements
and services provided by the Spoke to hosts behind the Spoke. The flexibility is a result of proper planning and placement
of Hub resources in logical locations within the network.
Intent Based policies

After a Spoke provisioning is complete you have the option of adding intent based policies to control traffic as it passes through the
Spokes. These are used to provide additional security services and enable devices on private address space to access public Internet.
Intents can be applied for either CSO SaaS or On-premise deployments using the CSO GUI.
Firewall Policies
Firewall policies are from source and can include address, device, department, zone or users. The destination can include devices,
addresses, zones, applications, services, Spokes or Spoke groups. Acceptable actions are allow, deny or reject. You can use a CSO
defined UTM policy or clone and modify as needed. This allows a user to block certain Spoke or type of Spokes at either a micro or
macro level.
Spoke-to-Spoke tunnels require Firewall policy to allow communication between Spokes. Firewall Intents are uni-directional but
stateful so the intent must be specified in both directions. This can be written into a single rule and deployed to target Spokes
simultaneously. Groups or departments can be used to deploy intents to all devices sharing the same object.
NAT
AutoNAT can be enabled during Spoke configuration and then all traffic is source NATted when egressing the Spoke or you have
the option to create and deploy NAT rules after the Spoke is provisioned using NAT Policy. Nat Policy allows an administrator to
specify the type of NAT (Source, Destination or Strict), and then specify source by address, zone, routing instance, protocols or
interfaces or any combination of sources. Destination can include addresses, zones, routing instances, services or interfaces. There
are also options for defining NAT Pools and Proxy-ARP.
SLAs
SD-WAN policy optimizes utilization of the WAN links and efficient load distribution of traffic. SD-WAN policy can be
applied to source endpoints (such as sites and departments) and destination endpoints (applications or application groups)
or for breakout traffic (by using breakout profiles). SDWAN policy can be defined by department, Spokes or group of
Spokes, associated with applications and service profiles. The profiles can be one of the breakout profiles or a link defined

SLA. Sla policies can be used to specify what applications are monitored and if the traffic fails to meet defined
characteristics the traffic can be moved to another link if available and the new link meets the defined characteristics.
Add Traffic profiles. Path steering profiles etc
Configuration Templates
On-Premise deployments can use CSO GUI to define and deploy intents or you can build Configuration templates. Configuration
templates are used to deploy additional settings to CSO managed devices. These can be organization type settings such log hosts,
Authentication servers, or SNMP configuration or they can be site specific. CSO allows admins to take any valid Junos CLI
configuration and create templates using Configuration Designer tool included with CSO. The templates can include variables to
allow changes on a per-Spoke basis. So, a template can be created and associated with a device template and then customized per-
Spoke. Configuration templates without variables can be automatically applied during later stages of provisioning or after
provisioning is complete. Configuration templates with variables appear as device configuration options and allow you to specify
parameters for the variables based on requirements of the individual Spoke.
CSO provides administrators a lot of options and flexibility in deploying Hubs and Spokes and Configuration templates provide a
means for additional device configuration through CSO provided templates or through custom Configuration templates.
Preparing for Deployment

Deployment preparation is of the utmost importance to ensure successful outcomes. We need to work together with the
customer to ensure that we understand their requirements and they understand CSO requirements. Below is a list of
questions designed to help understand customer requirements and expectations.
Determine the Deployment Type

It would be advantageous to have the customer put together a high-level description of their requirements so that we can
work with them to refine and create a design document. This will help Juniper understand their goals and environment and
we can then fine-tune to put together the low-level design document and worksheets.
Start by determining if this will be cloud delivered on an On-Premise installation. In addition to the questions in the SD-
WAN Deployment Considerations section, we also need to ask:
• Do they have a document outlining the services they plan to offer or provide using SD-WAN?
• Will this be an OTT build or all on customer network?
• Total number of Spokes and what type of CPE?

• Total number of Hubs and gateways

• Do you have public or reachable IP addresses set aside for Provider and Enterprise Hubs.
• Is HA required at server / Hub / gateway / Spoke.
• Do they have personal with skill set to configure and manage servers?
• Are there network personnel involved in project with access to underlay for configuration and troubleshooting if
needed?
• Will this be green field or brown field installation?
• Will existing WAN links be used or do they need to order new ones?
• Are there Mist Apps that are part of solution?
Use the decision chart to determine if the CSO 5.2 SaaS or On-Premise solution will fit their requirements.
In addition to qualifying the deployment we need to understand if this will be a clean install or there are integration
requirements that will additional work effort. We need to ask:
• Do they have an existing Juniper deployment? if so what device types?

• Will sites have to be migrated over to SD-WAN or is this a greenfield deployment?
• How will the Juniper SD-WAN solution be integrated with customer network?
• Does the customer have standardized configurations that must be applied via Configuration Templates?
• Do they need to import routes from Datacenters or larger locations?
• Customer security requirements for On-Premise solution
o They are responsible for FW – do they have list of CSO required ports and has their FW been configured
and tested?
o On-Premise installations require root access to servers running Ubuntu 16.04 or ESXi 6.0.
o Do you have their Authentication Server, log server, SNMP server addresses and credentials /
communities?
• Is the underlay configured and validated? Please provide a topology drawing.
• If existing deployment do you have remote access to the devices?
CSO requires certain devices for the various functionality. Use the customer’s high-level drawing to determine what
devices are needed and how many of each type. Once the topology drawing is complete then you can determine what
devices are required at each location.
• Do you have SRX1500/4100/4200 for hubs?

• How traffic will be broken out locally?
• What IPsec throughput is required?
• A dedicated vSRX or physical SRX to NAT to CSO servers and VRR is required for On-Premise installations.
• How many interfaces per Hub or Spoke?
• What type of Spokes will be used?
Create Comprehensive Design Document

Once the deployment type has been determined we can start working on a low-level design. The goal is to ensure all the
necessary equipment, personal, access, underlay etc. requirements are met.
The customer design document should be refined to specify the number of VLANs required on each node, a port to VLAN
mapping, an IP address plan, security/load balancing, WAN access, and other details required for each environment.
Please answer the following questions:

• Contact info for relevant parties.

• Do they have access to download Juniper software?
• Do they have customer account with Juniper?
• Do they have an existing Juniper deployment? if so what device types?
• 3rd part equipment as part of underlay? Who will have access and ability to troubleshoot?
• Please complete worksheet with:
o Device hostnames and naming conventions
o Public ip addresses set aside for data Hubs and gateways.
o DNS address
o NTP server address
o Addresses and credentials for any adjacent switches and routers
• RBAC – admin and operators identified?
• Account access to nfxweb.juniper.net for On-Premise installations.
• Do they plan on using one of the breakout options?
• Are there any additional configuration changes that must be applied via Configuration Templates?
o If so, please detail and send Junos CLI snippet.
• Is UTM required? If so, do they have proper licensing?
• Is NAT required? Is the design done?
• Do you have console access to devices?
On-Premise Installation
CSO servers should have a base operating system (Ubuntu 16.04.5 or ESXi 6.0) and meet the server specifications for the
intended installation type (small or HA Large). Server specifications can be found in the CSO Installation and Upgrade
guides.
Prior to beginning installation:
• Servers should be able to reach internet

• DNS / NTP configured on servers if you don’t have other servers providing those functions.
• Root access credentials (only needed for install)
• Support.juniper.net access to download CSO installation and device software packages.
Site Readiness
• Racks, power, cable, access, servers with correct code version installed.
• Underlay test and validated.
• All Hub-type devices connected and have network reachability
• Login credentials for Hub-type devices.
• Personal available to assist with site and underlay access as required.
Spending the time preparing for the installation by taking the time to understand customer requirement and their
environment will reduce the time to operationalize the SD-WAN deployment. We recommend documenting topology,
access methods, IP addresses, accounts, procedures for adding Spokes and Hubs, adding policies etc.
Conclusion

Contrail Service Orchestration designs, secures, automates, and runs the entire service life cycle across NFX Series
Network Services Platforms and SRX Series for NGFW or SD-WAN. It’s also a service orchestrator for the vSRX Virtual
Firewall, available in public cloud marketplaces such as Amazon AWS.
Contrail Service Orchestration combines innovative capabilities:
SD-WAN as a growth platform—Contrail Service Orchestration allows you to chart a course to SD-WAN and beyond by
seamlessly integrating Zero Touch Provisioning, full-stack security, monitoring.
Enabling automation to help simplify your operations, providing reliability and agility while extending visibility across your
multicloud network.
Pervasive, always-on security—Managing your WAN edge policy at scale and end to end from the cloud to the branch,
campus, and data center is all controlled through Contrail Service Orchestration. In addition to connecting, scaling, and
securing WAN topologies and services, Contrail Service Orchestration directs application-aware handling and deep security
inspection, enforcement, and analytics across all managed devices.
Terminology
Term Definition
Application Quality of Experience provides real time monitoring of traffic flows using active and passive
probes to measure application traffic for SLA compliance. CSO uses inline passive probes sent in
AppQOE conjunction with application traffic to monitor the link. CSO monitors other link candidates with active
probes in the event traffic on active link fails SLA and if a candidate links meet SLA requirements then
traffic is moved to candidate link
Branch A tenant site connected to other sites in either a Dynamic mesh or Hub-and-Spoke topology.
Dynamic Virtual Private Network. When traffic is seen by CSO between Spokes that meet administrator
DVPN
defined thresholds then CSO will provision an Ipsec tunnel between the Spokes.
A tenant site that acts as a Hub for traffic from multiple Spokes in a Hub-and-Spoke topology. In this
Hub
topology, all Spoke-to-Spoke traffic flows through the Hub.
Intrusion Detection / Intrusion Protection Systems - Monitor and identify unauthorized access or other
IDP / IPS
incidents then notify administrators and stopping the incident.
LBO Local Break Out – default route will be local next-hop not across VPN
MP-BGP Multiprotocol BGP; a routing protocol used for large-scale, multi-tenancy deployments.
Next Generation Firewall – Standalone Firewalls with advanced features managed and monitored by CSO
NGFW but are standalone devices as they do have connections to hubs for data termination. They will have
outbound-SSH for OAM functionality.
Site Any Enterprise customer branch offices or headquarters. Commonly referred to as Hub site or Spoke site.
Spoke A tenant branch site in a Hub-and-Spoke topology.

Typically, an Enterprise customer with many branches (sites) who subscribes to the SD-WAN offering
Tenant
provided by the provider.
Unified Threat Management provides additional security services that may include Web Filtering, Antivirus,
UTM
and Content Filtering.
VNF Virtualized Network Function – virtual router or other function that run on a virtual machine.
ZTP Zero touch provisioning, allow provisioning of devices automatically with minimal intervention.

Contrail SD-WAN Solution Design Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Contrail SD-WAN Solution Design Guide

Uploaded by

Copyright:

Available Formats

JUNIPER NETWORKS CONFIDENTIAL

CONTRAIL SD-WAN SOLUTION DESIGN GUIDE

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 2

Active/Active or Active/Backup ........................................................................................................................................................... 26

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 3

Determine the Deployment Type ............................................................................................................................................................. 38

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 4

Document Version Control

Version: Version 1v31

Date: July 17, 2020

Version Number Author Date Reason for Change

1.0 Andy Lupher 5-02-19 • Migrate 4.1 doc to incorporate 5.0

1.v5 Mark Scherbring 5-06-19 • Comments and additional material

1v31 Andy Lupher 7-17-20 • Updated for 5.3 CSOaaS

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 5

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 6

Figure 1 Topology view over SD-WAN Overlay

SD-WAN Deployment Considerations

What is the motivation for SD-WAN?

Increased bandwidth at lower cost

Unable to buy a single link with required capacity

Are you a managed services customer?

What types of branch CPE devices are needed?

What network services run at the branch?

How do you currently manage and operate the branch network?

How do you order WAN links to branch’s?

Do you need dynamic policy-based routing over multiple links?

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 7

Do you need to connect to AWS VPC or other cloud provider?

Do you need Application Aware Security?

Does your CPE choose best path based on application?

How will you prefer to monitor the SD-WAN network?

What reporting functions do you required?

A lot Size of network

Enterprise Purchase or Rent Yes Few

© 2018 Juniper Networks J3.02.P01.T07 Rev 17 Figure 2 Decision Flow Chart

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 8

Contrail SD-WAN Solution Overview

1. Zero touch device provisioning (ZTP) and cloud-based device management.

CSO Service Types

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 9

Enabling Multi-Tenancy and Segmentation.

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 10

Figure 3 Network Slicing for Multi-Tenancy

Figure 4 Network Routing for Single Tenant

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 11

CSO Overlay (Tunnels) Network

Figure 6 Cloud Delivered Branch

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 12

Operations and Maintenance (OAM)

2 Device contacts CSO

4 device establishes Ipsec tunnel to

5 CSO configures additional Ipsec

S-S MPLS tunnels

Figure 7 ZTP Provisioning Process

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 13

These rules apply to mesh tags:

• Spoke sites support one mesh tag per link.

Figure 7 illustrates some of these use cases.

JUNIPER NETWORKS CONFIDENTIAL © Juniper Networks, Inc. 14

Connecting different underlay links Site-to-site tunnels based on capacity

WAN_0 Overlay tunnel on

Geo based meshing With Dual CPE

© 2017 Juniper Networks, Inc. All rights reserved.