Aruba SD-WAN Integration With Public Cloud (AWS) - Technical Note

Aruba SD-WAN Integration with
AWS Public Cloud

Version 1.0
Author:
Samuel Pérez Buñuel
Contributors:
Laura Neacsu
Mani Ganesan
Shashikala Subramanya
Technical Note
Copyright Information
Copyright © 2019 Hewlett Packard Enterprise Development LP.
Open Source Code

This product includes code licensed under the GNU General Public License, the GNU Lesser General Public License,
and/or certain other open source licenses. A complete machine-readable copy of the source code corresponding to such
code is available upon request. This offer is valid to anyone in receipt of this information and shall expire three years
following the date of the final distribution of this product version by Hewlett Packard Enterprise Company. To obtain
such source code, send a check or money order in the amount of US $10.00 to:
Hewlett Packard Enterprise Company

Attn: General Counsel
6280 America Center Drive
San Jose, CA 95002
USA
www.arubanetworks.com
3333 Scott Blvd
Santa Clara, CA 95054
Phone: 1-800-WIFI-LAN (+800-943-4526)
Fax 408.227.4550
Contents
Contents
Revision History ................................................................................... 4

1 Overview .......................................................................................... 5
1.1 vGW in Public Cloud Infrastructure ............................................................................. 5
1.2 Virtual Gateway Orchestrator ..................................................................................... 6
1.3 AWS Connectivity Modes ............................................................................................. 8
2 Establishment of the Overlay Network .......................................... 9

2.1 Aruba SD-WAN Orchestrator Overview ...................................................................... 9
2.2 Configuration of SD-WAN Orchestration .................................................................. 13
2.3 Expected Result.......................................................................................................... 16
3 Single VPC deployments ............................................................... 18

3.1 Single VPC Architecture Overview ............................................................................. 18
3.2 Configuration for Single VPC Deployments .............................................................. 22
3.3 Expected Result.......................................................................................................... 36
4 Multi-VPC Deployments ................................................................ 39

4.1 Multi VPC Architecture Overview .............................................................................. 39
4.2 Configuration for Multiple VPCs ................................................................................ 42
4.3 Expected Result.......................................................................................................... 53
5 Annexure—Useful AWS Procedures ............................................ 56

5.1 Basic AWS Setup ........................................................................................................ 56
5.2 Setting up a Test Server in AWS ................................................................................ 65
6 Reference ...................................................................................... 69
Aruba SD-WAN Integration with AWS Public Cloud Contents | 3

Revision History Revision History
The following table lists the revisions of this document:
Revision Change Description
Version 1.0 Initial Publication
Table 1 Revision History
Aruba SD-WAN Integration with AWS Public Cloud Revision History | 4

1 Overview
This Technical Note describes how to integrate a Private Cloud infrastructure hosted in AWS into an Aruba
SD-WAN. In order to do so, the SD-WAN solution makes use of the Aruba Virtual Gateway (vGW), which will
serve as entry-point into the Virtual Private Cloud (VPC) environments a given customer may have in AWS.
Before we go into the details about how the Aruba vGW is deployed and operated, we need to set a basic
understanding about the different modes of operation within AWS.
1.1 vGW in Public Cloud Infrastructure

Aruba Branch Gateways support standard IPSec tunnels, and could therefore establish direct communication
with the AWS VPN Concentrators. So, why would an Aruba Virtual Gateway be needed to integrate with AWS?
The AWS VPN Concentrators don’t support the SD-WAN capabilities of an Aruba VPNC (or vGW, in the case of
Public Cloud infrastructure). The most critical would be:
• Reverse Path Pinning, which ensures the traffic always returns through the same path of origin as
it came from, allowing for BGWs to perform Uplink load-balancing and Dynamic Path Steering.
• Orchestrated Tunnels, which automates the establishment of IPSec tunnels from all BGWs to all
relevant VPNCs (including the vGW).
• Orchestrated Routing, which automates the exchange of routes across the SD-WAN.
• End-to-End visibility of all the SD-WAN network under a single application (Aruba Central).
Aruba SD-WAN Integration with AWS Public Cloud Overview | 5

In summary, the use of the vGW to connect the SD-WAN environment to the Public Cloud environment is
highly encouraged, as it truly brings the Public Cloud environment into the SD-WAN network as if it were
another Data Center.
Figure 1 - Role of vGW
1.2 Virtual Gateway Orchestrator

The Public Cloud environment is, for many companies, a “foreign” element to their network, where services
are brought up relying on tools provided by the cloud provider that are not necessarily similar to those in the

DC. For that reason, something more advanced than a simple VM offered through the marketplace is
desirable.
The Aruba SD-WAN solution can make use use of a software-as-a-service component of Aruba Central to
handle the full life cycle of the vGW (manual bringup is also supported).
This Technical Note describes the solution using the orchestrated workflow, as it’s generally the preferred
mechanism.
Figure 2 - vGW Orchestrator Functions

As highlighted in the image above, Aruba Central handles the whole lifecycle of the vGW, from the initial bring
up and provisioning, through the regular management (as if it were another VPNC in the network) and to even
handle failover in HA scenarios.
In order to do so, it connects with the Customer AWS account to have visibility over it and provision the VM,
together with the necessary interfaces, subnets, elastic-IP mappings, etc. The functions of the vGW
Orchestrator and the configurations required are covered in detail through this document.

1.3 AWS Connectivity Modes
From the perspective of the SD-WAN network, AWS deployments can be differentiated between those where
each VPC would be treated as a separate node of the SD-WAN (this generally happens when there is one or
just a few VPCs), and those where there are multiple VPCs accessible through a single SD-WAN node (generally
happens when there are multiple VPCs).
Figure 3 - AWS Deployment Models

This document will cover the deployment of an Aruba SD-WAN network integrating to single VPC
environments as well as to an AWS environment with multiple VPCs.

2 Establishment of the Overlay Network
Even though it’s not strictly the objective of this Technical Note, the main purpose of a vGW in AWS is to bring
the public cloud environment into the SD-WAN network. A brief introduction to the establisment of the SD-
WAN network is therefore needed.
In this sense, vGWs will behave just as any other headend gateway (also referred to as VPNC) in Aruba SD-
WAN. The solution will bring up tunnels through all WAN circuits to form a transport-independent secure
overlay. The Aruba SD-WAN solution supports two mechanisms for doing this:
• Manual overlay, where BGWs are configured to establish tunnels to a certain set of VPNCs and
share branch subnets as part of the tunnel negotiation.
• Orchestrated overlay, where headend and BGWs share their uplink and routing information with
the SD-WAN Orchestrator service, and automatically receive the tunnels and routes to connect to all
desired destinations.
The recommended mechanism to establish tunnels to an AWS (orchestrated) vGW is to use the orchestrated
overlay; as it enhances the HA capabilities by leveraging the high-priority control channel used by the SD-WAN
Orchestrator. This Tech Note will therefore focus on the Orchestrated overlay.
2.1 Aruba SD-WAN Orchestrator Overview

A key component of the Aruba SD-Branch solution is the SD-WAN Orchestrator included as a SaaS application
in Aruba Central. This is a horizontally scalable SaaS that performs two main tasks:
• Orchestrate tunnels through all circuits to form a secure overlay network.
• Orchestrate the sharing of prefixes across the SD-WAN to remove the need to run traditional routing
protocols between the nodes of the SD-WAN.
If you’re already familiar with how the SD-WAN Orchestrator works, feel free to skip this part and
NOTE
go directly to section 3.
Aruba SD-WAN Integration with AWS Public Cloud Establishment of the Overlay Network | 9
2.1.1 Control Channel Communication
The communication between the Aruba SD-WAN gateways (regardless of their function in the network) is
established through a gRPC (refer to grpc.io) channel that Gateways use to communicate with the SD-WAN
Orchestrator (over TCP 443). This is a high priority communication channel that is established through any of
the available uplink interfaces of any gateway.
Figure 4 - SD-WAN Control Channel
2.1.2 Tunnel Orchestration
In order to build an SD-WAN network, the first step is to bring up a transport independent secure overlay
network. Or, in other words, manage the establishment of IPSec tunnels between the nodes of the network
through all available circuits as per the policy defined by the administrator.
In order to do this, the administrator will identify the uplink interfaces in all nodes with the corresponding
service provider. Once that’s available, the SD-WAN Orchestrator instructs the establishment of tunnels
according to the policy. This will be done by matching the service provider uplinks in the following way:
• In the case of private circuits (MPLS), the name (i.e., SUPER) will have to match on both ends. Partial
matches such as “SUPER01-MPLS” with “SUPER_MPLS” would also result in sending IPSec SAs to both
ends of the tunnel.
• In the case of public circuits (INET, MetroE, LTE), the SD-WAN Orchestrator will first try to establish
tunnels between uplinks with matching names (i.e., SPEEDY_INET with SPEEDY_INET). If no match is
possible, cross-SP matches will be attempted (i.e., SPEEDY_INET with FAST_INET). In both cases, the
SD-WAN Orchestrator will send the IPSec SAs to the gateways to bring up the tunnels.
Figure 5 - SD-WAN Tunnel Orchestration

In the case of the vGWs in AWS, the tunnels will come through the interface connected to the VPN Gateway
(Direct Connect coming from the customer’s private network, if at all in use) as well as through the interfaces
connected to the Internet Gateway (ref: Figure 1 - Role of vGW). The WAN circuits should therefore be labelled
as MPLS for the one connected to the VPN Gateway and INET for the one connected to the INET Gateway.
2.1.3 Route Orchestration
The last component for the SD-WAN Orchestration is the automation of routing between the nodes in the SD-
WAN network. For this purpose, the SD-WAN Orchestrator operates like a horizontally scalable routing service
for the SD-WAN:
• Routes learnt by the different gateways participating in the SD-WAN network will be advertised (or
redistributed) into the SD-WAN.
• Likewise, routes learnt via SD-WAN will be redistributed or advertised to other routing protocols.
• Route summarization inside the SD-WAN will be done by the route orchestrator itself.
• The summary routes needed for branch-to-branch communication through the hubs will be
selectively advertised as per configured policy.
• When a given prefix is reachable through multiple paths, set cost as per the configured policy.
Figure 6 - SD-WAN Route Orchestration
2.2 Configuration of SD-WAN Orchestration
The configuration for SD-WAN orchestration is extremely simple, consisting of only three steps:
• Labeling WAN interfaces
• Establishing DC preference
• Redistributing routes
This is covered in greater detail in the SD-WAN product documentation. Nevertheless, the minimum steps are
outlined below for the sake of completeness.
2.2.1 Define WAN Uplinks

The first step is to define the uplink interfaces and assign a label to them. This can be done at the group or
device level in the case of BGWs, but must be done at the device level for Headend Gateways (VPNCs).
Gateway Management  WAN  Uplink
Figure 7 - Define WAN Uplinks
2.2.2 Establish SD-WAN Overlay

The next step consists in enabling SD-WAN Orchestrator for VPNC and Branch Gateway groups. This
configuration should be done at the group level.
Gateway Management  VPN  SD-WAN Overlay
DC preference for branch groups will be defined in this section:
Figure 8 - SD-WAN DC Preference – Branch configuration

For VPNC groups, DC route summarization as well as whether to allow branch to branch traffic through the
hub will be defined (generally not the case for vGWs, as there are costs associated with the network traffic
egressing the VPC):
Figure 9 – SD-WAN Orchestration-DC Aggregate routes
2.2.3 Redistribute to/from Overlay
As mentioned above, the last step will be to redistribute other routing protocols (and connected subnets) into
SD-WAN and vice-versa.
Figure 10 –Redistribute into SD-WAN
2.3 Expected Result
Once the configuration is done, the outcome can be easily verified from the Aruba Central monitoring screens.
2.3.1 Tunnel Orchestration

Orchestrated tunnels can be verified from “Monitoring & Reports > SD-WAN Orchestrator > Overlay Tunnel
Orchestrator”. Ensure that the control connections are UP, then check the status of the tunnels:
Figure 11 - SD-WAN Tunnel Orchestration – Connections
Figure 12 - SD-WAN Tunnel Orchestration – Tunnels
2.3.2 Route Orchestration
Once the tunnels are up, check that the routing is working as expected. On the “Monitoring & Reports > SD-
WAN Orchestrator > Overlay Route Orchestrator” dashboard, check the routes advertised by the different
nodes of the SD-WAN.
Figure 13 - SD-WAN Route Orchestration Learned/Advertised Routes

Lastly, the Gateway Details page displays the final list of routes that are learned/advertised from/to SD-WAN.
Highlighting the routes that will be installed in the routing table.
Figure 14 - Routes Learnt from SD-WAN
3 Single VPC deployments
3.1 Single VPC Architecture Overview
3.1.1 Connectivity into the VPC
For an SD-WAN environment where the BGWs are connected to the AWS environment through one or
multiple Internet circuits or a combination of Internet circuits with an MPLS with AWS DirectConnect, the
deployment would look more or less like the illustration below:
Figure 15 - SD-WAN with Multiple INET Circuits Figure 16 - SD-WAN with Combination of MPLS and
INET
From the perspective of the SD-WAN (and the SD-WAN vGW), the only difference between these two models
(Internet only or combination of MPLS and Internet) is the fact that tunnels would be just going through the
Internet, or through the Internet as well as the DirectConnect interface.
Aruba SD-WAN Integration with AWS Public Cloud Single VPC deployments | 18
3.1.2 Routing inside an AWS VPC
In a single VPC environment, the only routing mechanism available in AWS is to apply routing tables to the
subnets in the VPC (source: AWS). These routing tables have a maximum size of 50 entries (source: AWS),
which means that learning all routes from the SD-WAN is not a good option. In order to connect this
environment to the SD-WAN and other resources, the vGW will act as the gateway for all traffic coming in and
out the VPC.
The vGW Orchestrator facillitates this connectivity by creating 8 /27 subnets to connect the vGW with other
resources in the VPC. The result will be:
• VPC subnets will point to the vGW for all traffic in/out of the VPC.
• The vGW should have routes pointing traffic destined to corporate subnets through the VPN
Gateway. Use larger subnets to those learnt from the SD-WAN to ensure that traffic always picks the
shortest path (as with any other router, the vGW will use the more specific routes).
• The vGWs default gateway will be the VPC’s Internet Gateway.
The resulting network diagram would be something like this:
Figure 17 - Routing Inside a single VPC
3.1.3 High Availability in Single VPC environment
As mentioned before, the native routing mechanism available inside a VPC are static routes applied to the
subnets where the workloads are connected. That, coupled with the fact that AWS doesn’t support broadcast
or multicast inside the VPC (source: AWS) makes High Availability (HA) quite challenging. Thankfully, the Aruba
vGW Orchestrator can take care of providing high availability in such environment.
The (HA) solution for Aruba vGW works in an Active-Passive paired configuration. In this setup, there can be
only one active Virtual Gateway that can forward data in and out of the Virtual Private Cloud's (VPC) subnets.
The Virtual Gateway Orchestrator app decides which of the Virtual Gateways becomes Active or Passive and
communicates this decision to all the pivotal components. The decision of setting the Active or Passive Virtual
Gateway is based on the following requirements:
• Virtual machine health of each of the vGWs in a given HA-pair.
• The health of the vGW gRPC control connectivity to the SD-WAN Orchestrator.
• The connectivity of each of the vGWs to all the BGWs.
The vGW Orchestration app decides the status based on the state of each Virtual Gateway. The app runs a
poll every 20 seconds on each vGW for these criteria:
• VM Health: The VM health status is obtained from the cloudapp (AWS) for each of the gateways in all
the VPCs across each of the regions of a given AWS cloud account.
• Control Health: The control health status is provided by the SD-WAN Orchestrator.
• Data and Tunnel Health: The data and tunnel health status is provided by the SD-WAN
Orchestrator module.
3.1.3.1 Failover of Subnet Routes

Once the vGW Orchestrator has brought up the vGWs in the VPC (ideally in multiple availability zones), the
VPC subnets will be pointing to the Active vGW:
Figure 18 - vGW HA - Initial status
In the event of a failover (which can be caused by any of the reasons mentioned above), the vGW Orchestrator
automatically modifies the routing table attached to the subnets to start pointing them towards the other
vGW, which will now be the active vGW.
Figure 19 - vGW HA - Failover
3.1.3.2 SD-WAN Failover

As mentioned above, the SD-WAN Orchestrator can trigger a failover when a partial outage is detected in the
active vGW. In such environment, the SD-WAN Orchestrator would work in conjunction with the vGW
Orchestrator to steer all traffic to the new active vGW.
Figure 20 - vGW Failover - SDWAN Routing
3.2 Configuration for Single VPC Deployments
3.2.1 Basic AWS Setup
Even though the vGW would generally be set up in an existing VPCs, knowing how to do at least a basic setup
will be necessary to have a good understanding about the overall solution.
The following are pre-requisites needed by the vGW Orchestration service:
• A non-default VPC must exist for the vGW Orchestration to work.
• The Orchestration engine will split it into 8 /27 subnets and consume them to connect the vGW to
different resources in the VPC. This means that the VPC must at least have a /24 subnet
available.
• Internet and VPN Gateways, as the vGW Orchestrator will connect the vGW to these AWS objects.
• A Security group to place the vGW into (the default group can be used, but it’s not advised). Ensure
that you enable inbound UDP 4500 for the IPSec tunnels to come up.
• An SSH Key Pair to allow an out of band connection into the vGW before it’s managed by Aruba
Central.
And the following would usually be in place (but aren’t mandatory):
• Subnets (with VMs connected to them)
• Route tables – In the case of AWS, route tables are directly tied to subnets
• Resources (Instances)
NOTE Step-to-step guidance on how to set up the VPNC can be found in the annexure of this document.
3.2.2 Virtual Gateway Orchestration
As mentioned in the earlier sections, Aruba’s SD-WAN solution handles the whole lifecycle of a Virtual
Gateway, including the orchestration of the vGW AMI. This will not just matter for the provisioning phase, but
it will also be critical to provide high availability in single VPC environments, where traditional L2 mechanisms
such as VRRP won’t work (source: AWS).
For this orchestrated workflow, Central uses a third-party token. This token is only valid when used from a
certain Aruba Central account:
3.2.2.1 Generate Third- Party ARN Token

To generate a third-party ARN token:
1. From your AWS console, go to Services > IAM > Roles > Create Role to generate the third-party token.
Figure 21 - Generate Third-Party Token
2. In Aruba Central, go to Global Settings > Virtual Gateway > Accounts > Add Account. Copy the
Account ID and External ID:
Figure 22 - Account Details

3. Select Another AWS Account. Enter the details provided by the vGW Orchestration app in Central. The
account ID identifies Aruba Central within AWS; The External ID identifies the Aruba Central customer
account that will use the token:
Figure 23 - Account ID and External ID
4. Set permission to be “AmazonEC2FullAccess”. This will allow Aruba Central to handle AMIs, subnets,
Elastic IPs, etc. Click Next: Tags, and then click Next.
Figure 24 - IAM Permissions
5. To generate the ARN, provide a name for the AWS role and click Create Role. This will provide the ARN
to be pasted in Aruba Central.
Figure 25 - IAM Role Creation
3.2.2.2 Virtual Gateway Orchestration
The third-party token can then be used to give Aruba Central access to the AWS account. This will allow the
Orchestration app to read the state of the VPCs and create/monitor/fail-over/delete the vGWs.
1. In Aruba Central, go to Global Settings > Virtual Gateway > Accounts > Add Account. Paste the
account name (which can be any administrative name we provide) as well as the third-party token ARN
(obtained from the AWS IAM workflow).
Figure 26 - Paste IAM role ARN

This will trigger the Orchestration app to connect to the AWS account. It should go to INIT status first and,
after a few seconds, to ACCESS VERIFIED status:
Figure 27 - Access to AWS Verification
2. As soon as the account is in ACCESS VERIFIED status, go to Deployment. The Orchestration app will
automatically display the VPCs in our account as well as the subnets belonging to the different VPCs.
3. It will also give us the option to spin up a vGW (or vGW pair in HA) in our VPC. Once there, simply
provide Virtual Gateway size, AWS Instance type, Key name, and Security group and click Deploy
Virtual Gateway. The deployment will take 1-2 minutes, during which the Orchestration app will
display a message saying DEPLOYING VIRTUAL GATEWAY and a progress bar.
Figure 28 - Create vGW
4. Once the deployment has finished, the message will change to “VIRTUAL GATEWAY DEPLOYED”.
Hovering over the message displays the serial number(s) of the Virtual Gateway(s) that have been
deployed.
Figure 29 - vGW Deployment finished

5. To monitor the status of the VM, go to the Summary tab. Once again, hovering over the different
values in the table provides additional information.
Figure 30 - vGW Summary

The vGW will need a few minutes to boot, but since all the configuration for Aruba Gateways comes from
Aruba Central, it’s advisable to start configuring it right away.
3.2.4 vGW Initial Configuration in Aruba Central
Now that the vGW AMI has been orchestrated, the vGWs can be configured like any other VPNC. The following
steps will therefore resemble those that we would follow with an on-premises VPNC.
3.2.4.1 Initial vGW Group Setup

Global Settings  Manage Groups  New Group
If not already available, create a new group for the vGWs. For the orchestrated HA mechanism to work, we
should have a dedicated group per VPC. Please note that the group password will be the default user admin
password for all the gateways in this group.
Figure 31 - Group Creation

A few other details to keep in mind are that we should have a single group per pair of vGWs for orchestrated
HA to work and that this TechNote describes the configuration done using the GUI. Therefore, create the vGW
group as a GUI group for Instant AP and Gateway.
1. Go to Gateway Management  click “Filter Gateway Management” from the top of the window 
select your vGW group under GROUPS.
2. Set the group type to VPNC:
Figure 32 - Group Type
3. Go to Global Settings  Manage Groups  Drag vGW to the newly created “Virtual Gateway” group.
Figure 33 - Move vGW to Group

As an alternative, if the Virtual Gateway is still booting and hasn’t contacted Aruba Central, this task may have
to be done from the “Device Inventory” page:
Global Settings  Device Inventory  Assign Group Assign Device(s):
Figure 34 - Assign vGW to group from Device Inventory
3.2.4.2 vGW Initial Configuration
The orchestrated nature of the vGW will mean that the Virtual Gateway group will have slightly different
characteristics that the ones we’d normally see in a VPNC group. Ports and VLANs will be automatically set by
the orchestration engine, and some system-wide parameters such as the System-IP work better when
automated.
3.2.4.2.1 Basic System Settings

As mentioned before, the System-IP is a critical configuration element for each Aruba Gateway operating as
a VPNC or BGW. Each Aruba Gateway uses one VLAN interface as its System-IP. By default, this interface is
used by the Aruba Gateway to communicate with network services such as RADIUS, Syslog, TACACS+, SNMP,
DNS or NTP.
Given the dynamic nature of the vGWs, it’s recommended to set their System-IP with the mechanism used for
BGWs:
• Create Gateway Pool from Gateway Management > Select your VPNC group > Interfaces > Pool
Management > Gateway Pool.
• Assign Gateway Pool to the System-IP VLAN from Gateway Management > Interfaces > VLANs:
 Enable Routing: checked 

 IP Assignment: gateway-pool – System-IP
 Force operational status UP: checked 
• Set System-IP VLAN from Gateway Management > Interfaces > VLANs.
Aruba Central does not push any configuration to a gateway until the System IP is present, which
NOTE means that, once you set the System IP, Aruba Central will push all pending configurations and
trigger a reboot.
Other recommended settings that can be done at the group level are:
• Disable Spanning Tree – as there’s no need for it inside an AWS VPC.
• The vGW will learn the AWS DNS server as part of the orchestration. To use a different one, set it
manually.
• Set an NTP Server
3.2.5 Virtual Gateway Routing Configuration
When using the Virtual Gateway Orchestration in Aruba Central, vGW interfaces will be automatically set with
an IP address. In the case of the INET port/VLAN, even the default-gateway will be learnt. The vGW
Orchestration app will create /27 subnets to connect to the IGW, the VPN GW and the VPC LAN. These subnets
will be associated with the following Ports/VLANs:
AWS Interface vGW VLAN vGW Port Connects to
aruba-vgw-<n>-eth1 4094 GE 0/0/0 Internet (IGW)
aruba-vgw-<n>-eth2 4093 GE 0/0/1 DirectConnect (VPN

GW)
aruba-vgw-<n>-eth3 4092 GE 0/0/2 VPC Subnets (not

yet…)
NOTE n will be 0 for the “active” vGW and 1 for the “passive” vGW when setting up redundant vGWs.
Figure 35 - vGW subnets
At this point, the AWS subnets still don’t have a route to the vGW. This step is achieved by clicking
NOTE
Connect in the vGW Orchestration page, this step should be done after vGW is configured.
3.2.5.1 VPNC Network Configuration (Device-specific)
Even though most of the configuration is done by the vGW Oorchestration app, it is still necessary to add
routes pointing to the VPC as well as to the AWS VPN GW to ensure the traffic gets routed properly. This can
be done through the device-specific configurations of each vGW by going to Gateway Management > vGW
(device-specific) > Routing > IP Routes”.
Considering the vGW Oorchestrator has used the A.B.C.0/24 subnet to provision the different interconnect
/27 subnets, the AWS “gateway” IPs will be the following:
AWS Interface INET GW (VLAN VPN Gateway (VLAN LAN Gateway (VLAN
4094) 4093) 4092)
Active vGW A.B.C.33 (learnt via A.B.C.65 A.B.C.97

DHCP)
Passive vGW A.B.C.161 (learnt via A.B.C.193 A.B.C.225

DHCP)
This can also be checked (once the vGW has come up), through the monitoring pages in Aruba Central.
Monitoring and Reports  Gateways  Select vGW  LAN
Figure 36 - IPs learnt from AWS
3.2.6 Connect Virtual Gateway to VPC Subnets
Once it’s verified that the vGW is UP and operational, it’s finally advisable to start routing traffic through it. To
do so, go to the orchestration app and click Connect in the subnets that need to be routed through the vGW
from the Deployment tab of the vGW Orchestration app.
Global Settings  Virtual Gateway  Deployment
Figure 37 - Connect Subnet to vGW
It’s important to keep in mind that this routing table will replace whatever was previously
CAUTION!
attached to the device. Ensure that your vGW has the ability to reach all destinations!
After this change, the topology of the AWS VPC would look more or less like this (simplified view):
Figure 38 - AWS topology - Single VPC
3.3 Expected Result

3.3.1.1 vGW Configuration Verification
Once the vGW is up and configured, the Aruba Central monitoring dashboards display details of the state of
the Virtual Gateways. To view a summary of the device status, click Overview. To view the subnet information,
click WAN and LAN tabs.
Monitoring and Reports  Gateways  Select vGW  Overview
Figure 39 - vGW Monitoring Information
To check the real-time routing information, go to Routing > Route Table:
Figure 40 - vGW Routing Information

Lastly, and as it happens with the BGWs, more advanced actions like rebooting the device or launching a CLI
session into the vGW can be triggered from the Actions dropdown, and troubleshooting workflows can be
accessed through the Open Tools button.
3.3.2 AWS Routing Configuration
Clicking Connect in the vGW Orchestrator workflow triggers a route being attached to the subnet in AWS
pointing all non-VPC traffic through the vGW. This can be further validated in the AWS console by going to
Services > VPC > Subnets:
Figure 41 - Subnet Routing Tables
4 Multi-VPC Deployments
4.1 Multi VPC Architecture Overview
4.1.1 Connectivity into the Cloud Environment
When bringing SD-WAN into a more advanced AWS environment with multiple VPCs, having vGWs in every
VPC is not a scalable model, nor it makes any operational sense. For such scenarios, Aruba recommends
setting up an “edge VPC” that would serve as the gateway between the AWS environment and the SD-WAN.
Aruba vGWs would be deployed in this “edge VPC” and would peer directly with the AWS Transit Gateway
(TGW) (detaied information about the AWS TGW can be found in this link).
Figure 42 - Overview Architecture with Multiple VPCs
Aruba SD-WAN Integration with AWS Public Cloud Multi-VPC Deployments | 39

The mechanism supported by AWS to exchange routes with the TGW is to set up IPSec connections between
the vGWs and the TGW and to run eBGP inside the tunnels. That allows the dynamic exchange of routes
between the SD-WAN and the AWS environments.
4.1.2 Routing between AWS TGW and Aruba vGW

The communication between the AWS TGW and the Aruba vGWs (and ultimately the SD-WAN network) is
established by bringing up redundant IPSec tunnels between the AWS TGW and the Aruba vGWs. Once the
tunnels are up, eBGP sessions would be established to exchange prefixes between the cloud environment
and the SD-WAN environment.
Figure 43 - Peering between vGW(s) and TGW

4.1.3 High Availability in Multi-VPC Environment
In the same way as it happens for the single VPC environment, the vGW Orchestrator, together with the SD-
WAN Orchestrator, will take an active role in the HA process. The HA solution works by pairing Active-Passive
vGWs and having the SD-WAN Orchestrator app prefer routing traffic through the Active vGW.
However, as opposed to what happens in a single VPC environment, when peering with the AWS TGW, the
SD-WAN vGWs use BGP to exchange routing information as well as to handle failover. This means that, in
order to keep a symmetric communication flow, the “passive” vGW should be configured to advertise routes
that look “less attractive” to the TGW.
In this case, the AWS recommendation is to use AS_PATH to influence the routing decisions of the TGW
(source: AWS). Advertising a longer AS_PATH from the Passive vGW will influence the TGW to choose as best
path the one advertised by the Active vGW, as it has a shorter “AS_PATH”.
The recommendation is, therefore, to prepend the passive vGW’s own AS in the advertised AS-PATH when
advertising SD-WAN prefixes to the the TGW.
Figure 44 - Multi VPC Routing Failover
4.1.4 Route Summarization

It’s generally recommended to summarize routes when operating in medium to large SD-WAN environments
to avoid saturating the routing tables of the smaller gateways. In the case of the integration with the AWS
TGW, this recommendation can almost be considered a mandate, as the AWS TGW can learn a maximum of
100 BGP prefixes per IPSec VPN attachment (source: AWS).

4.2 Configuration for Multiple VPCs
As mentioned above, an edge VPC (also referred to as “transit VPC”) would have to be created to place the
vGWs. The same steps described in the section about the Basic AWS Setup can be followed to create such
VPC. Once the edge VPC is ready, the orchestration of the vGWs can be done exactly as described in the
section about the Virtual Gateway Orchestration. Finally, the initial configuration of the vGWs, would be done
as described in the section about vGW Initial Configuration in Aruba Central.
4.2.1 Transit Gateway Setup and Integration

In multi-VPC scenarios, the TGW will most likely be present at the moment of the SD-WAN integration.
Nevertheless, it’s important to understand the mechanisms involved, as well as the steps to make it happen.
4.2.1.1 Create the TGW

Go to Services > VPC >Transit Gateway and click Create Transit Gateway:
Figure 45 - Create AWS TGW

4.2.1.2 Create the TGW attachments
The TGW will then need to have VPC attachments between the different VPCs in the AWS environment. Go to
Services > VPC > Transit Gateway Attachments > Create Transit Gateway Attachments (for more
information, refer to the AWS documentation).
Figure 46 - Create VPC Attachments in AWS TGW

The solution will also require VPN attachments between the TGW and the vGWs. The public IP address
assigned to each vGW as part of the orchestration process will be needed for this configuration:
Figure 47 - VPN Attachment to TGW

This will result in a set of site-to-Site IPSec tunnels created. The configuration for vGWs can be downloaded
from the AWS console:
Figure 48 - TGW Site-to-Site VPN tunnels

On clicking Download Configuration, AWS provides a config snippet to be pasted in the customer gateway
(in this case the vGW). The Aruba vGW is configured by means of a GUI, so there won’t be a CLI output to copy
and paste. Download the Generic template to gather the necessary information:
Figure 49 - Download Vendor-Generic IPSec Template
The configuration downloaded from AWS includes information for the setup of 2 IPSec tunnels
NOTE
(and 2 BGP peerings). Don’t miss the configuration for the second IPSec tunnel/BGP peering.

4.2.2 Configure the Aruba vGW
The information downloaded from AWS will then be used to configure the vGWs.
4.2.2.1 Create the eBGP Peering Interface VLANs

The vGW will need an IP address to establish the BGP peering with the TGW. If the default options are selected,
AWS suggests the use of link-local addresses (169.254.x.y/30 subnet provided by AWS) that will be used. Since
these will only be used for the p2p communication between vGW and TGW, making use of these subnets is a
perfectly valid choice.
These IP addresses will be configured on an “always UP” VLAN interface (set force operational status to UP)
or on a loopback IP address. Make sure you configure the interface IP/netmask exactly as provided by AWS.
Figure 50 - Peering VLAN(s) IP configuration
NOTE Remember, this step has to be done for both IPSec tunnels.

4.2.2.2 Set up the IPSec Tunnels
The next step would be to create the IPSec tunnels between the vGWs and the TGWs. For this setup, once
again, make use of the information downloaded from AWS.
The following are the recommended parameters for source/destination network:
• Source network/mask: Inside <Customer Gateway IP>/32

• Destination network/mask: Inside <Virtual Private Gateway IP>/32
Figure 51 - IPSec Tunnel Source/Destination

Where the IKEv1 policy is defined as AES128 – SHA with DH group 2:
Figure 52 - IKEv1 Policy

The minimum recommended transform-set is esp-aes128-sha. More robust transform-sets like “default-aes”,
with aes-256-sha have also been validated.
Figure 53 - Set Transform-Set

For the rest of the tunnel parameters, the following are the recommended parameters:
• Peer Gateway IPv4: Outside Virtual Private Gateway
• VLAN: 4094
• Trusted, Enforce NAT-T, Pre-Connect, Force tunnel
• PFS Group 2

Figure 54 - IPSec Tunnel Parameters

4.2.2.3 Configure BGP
The last step is to configure the vGWs to establish an eBGP peering between the vGWs and the TGW to start
exchanging prefixes. This configuration will be based on the information downloaded from AWS:
Figure 55 - BGP configuration

Where allowall route-map is matching all prefixes received or advertised by the peer:
Figure 56 - Route-map Applied to BGP Neighbor

Finally, redistribute SD-WAN Overlay routes into BGP. At this point, make sure the “Passive” gateway is
advertising a longer AS-PATH by prepending its own AS.
Figure 57 - Redistribute SD-WAN to BGP

You can also verify if the BGP-learnt routes are advertised into SD-WAN:
Figure 58 - Redistribute BGP into SD-WAN
4.2.2.4 Route summarization

For most deployments, an additional step would be required; that is, to enable route summarization. This is
necessary to ensure that the BGP routing table in the TGW is not overflowed by the prefixes advertised by the
SD-WAN. As with the case of the redistribution, ensure you include AS-Prepending to make prefixes advertised
by the Passive vGW less attractive to the TGW.
Figure 59 - Aggregate BGP advertisements

4.3 Expected Result
After the tunnels and routing are configured, both AWS and Central will start reporting that the tunnels are
UP and that the routing information is being exchanged.
4.3.1 Expected Result in AWS Console

A few seconds after the tunnels and BGP peerings have been established, the AWS console will start displaying
tunnel status as UP (and the TGW is learning BGP routes from the vGWs). This can be monitored through
Services > VPC > Site-to-site VPN Connections.
Figure 60 - AWS Site-to-Site VPN

At the same time, the TGW routing rable will start being populated with routes coming from the SD-WAN
network. This can be monitored through Services > VPC > Transit Gateway Route Tables.
Figure 61 - TGW Routing table

4.3.2 Expected Result in Aruba Central
Aruba Central displays the tunnels established between the vGWs and the AWS TGW. In order to check this,
go to the Gateway Details dashboard and click the Tunnels tab:
Figure 62 - vGW Tunnel Status

Once the tunnels are up, the next step would be to check the routing information. This can be viewed in
Routing > BGP or Routing > Route Table.
When debugging BGP issues, ensure that the TGW appears as “Established” in the neighbors table and that
the vGW is receiving “paths” from it:
Figure 63 - BGP Neighbors

Next, verify that the prefixes learnt from the TGW are installed in the BGP routing table:
Figure 64 - BGP Routing table

And, lastly, double- check that the routes learnt from the TGW are making their way to the routing table:
Figure 65 - vGW Routing Table

To view more information about the routes, you can use the the troubleshooting workflows in Aruba Central
or through the Aruba Central remote console.

5 Annexure—Useful AWS Procedures
5.1 Basic AWS Setup
The steps to set up a test VPC are described in the following sections:
5.1.1 VPC Creation

There are several ways to set up a VPC. The following steps describe how to manually create a VPC:
1. From the AWS regional console, go to Services > VPC > Your VPCs, and click Create VPC:
Figure 66 - VPC Creation

2. Fill the name and addressing space to be used:
Figure 67 - VPC Network
Aruba SD-WAN Integration with AWS Public Cloud Annexure—Useful AWS Procedures | 56
5.1.2 Internet Gateway (IGW)
Follow the steps described in this section to create an Internet Gateway, (which is the AWS object that connects
the VPC to the outside world):
1. Go to Services > VPC > Internet Gateways and click Create Internet Gateway”
Figure 68 - Create IGW

2. Provide a name for the IGW:
Figure 69 - IGW Creation
3. Attach it to the recently created VPC:
Figure 70 - Attach IGW to VPC
5.1.3 VPN Gateway
The next steps are to be followed to create a VPN Gateway (which is the AWS object used to set up a
DirectConnect between VPC and the customers’ private network):
1. From the AWS regional console, go to Services > VPC > VPN Gateways:
Figure 71 - Create VPN GW

2. Set the name for the VPN Gateway:
---
Figure 72 - Name VPN Gateway
3. Attach the VPN Gateway to the recently created VPC:
Figure 73 - Attach VPN GW to VPC
5.1.4 Create a Security Group
The next steps are to be followed to create a Security Group. This is the firewall policy applied to a given AWS
instance. In the case of the vGW, we need to ensure that inbound port UDP 4500 is open to allow tunnels
coming from the Branch Gateways. From the regional AWS console, go to Services > VPC > Security Groups
and click Create security group.
Figure 74 - Create Security Group

1. Give it a name and attach the security group to your VPC:
Figure 75 - Name Security Group
2. Create inbound policies. Make sure you at least allow inbound UDP 4500.
Figure 76 - Customize Security Policy
5.1.5 Create SSH Key Pair
The last of the mandatory steps would be to create a SSH Key pair. In case anything were to go wrong with
the orchestration, this Key Pair can be used to SSH into the vGW.
Once this is done, the AWS console provides a PEM key that will allow user to SSH into our vGW (provided the
security policy allows inbound SSH). Make sure you keep it in a secure location.
From your AWS regional console, go to Services > EC2 > Key Pairs and generate your Security Key:
Figure 77 - Generate Security Key
5.1.5.1 Create a Test Subnet
Although not strictly necessary, having a test subnet is usually helpful. The last step would therefore be to
create a test subnet where test services can be brought up:
1. Create the subnet:
Figure 78 - Create Test Subnet

2. Assign name and addressing. Attach to VPC:
Figure 79 - Name Test Subnet and Attach to VPC
5.2 Setting up a Test Server in AWS
If you have been following the steps described in this Technical Note, the communication with the AWS VPC
should be all set. However, validating operational status using only monitoring dashboards in Aruba Central
may not be sufficient. Here’s a small guide explaining how to bring up a small Linux server in AWS to connect
to it through the SD-WAN and test the service end-to-end.
5.2.1 Bringing up a Linux VM in AWS

Bringing up a small Linux server in AWS is extremely straigthforward. This procedure includes the following
steps:
1. From your AWS regional Console, go to Services > EC2 > Instances and click Launch Instance:
Figure 80 - Launch AWS Instance

This leads to a menu to select the instance.
2. Select the first one from the list:
Figure 81 - Amazon Linux Server
3. Select the t2.micro VM and click on Next: Configure Instance Details:
Figure 82 - Select VM
4. If this server is purely being brought up for testing purposes, select Request Spot Instance (it’s
significantly cheaper, as it doesn’t have reserved resources). More importantly, make sure you the right
VPC and subnet. Click Review and Launch.
Figure 83 - Launch Test VM
You’ll then be given the option to use a Key Pair or use one that has been previously generated.
5.2.2 Connecting to the Test Server
Now that the test server is up, connect to test server through SSH from the test branch network. In your AWS
regional console, go to Services > EC2 > Instances. Click on the checkbox next to your test server instance
and click Connect.
Figure 84 - Connect to Test Server

The console displays a screen explaining how to SSH into the test server:
Figure 85 - Connect Using PEM Certificate
6 Reference
Aruba SD-Branch Fundamentals Guide:
• https://community.arubanetworks.com/t5/Validated-Reference-Design/SD-Branch-Fundamentals-
Guide/ta-p/482038
Mid-Size Deployment Guide:
• https://www.arubanetworks.com/assets/tg/AVD_SD-Branch-Midsize-Design.pdf
Aruba SD-Branch Online Documentation
• http://help.central.arubanetworks.com/latest/documentation/online_help/content/sd_wan/cloud_ga
teway/vgw_overview.htm
• https://help.central.arubanetworks.com/latest/documentation/online_help/content/gateways/cfg/ov
erlay-orchestration/sdwan-oto-oro.htm
AWS links:
• VPC Route tables: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Route_Tables.html
• Amazon VPC limits: https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html
• Amazon VPC FAQs: https://aws.amazon.com/vpc/faqs/
• Transit Gateway limits: https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-limits.html
Aruba SD-WAN Integration with AWS Public Cloud Reference | 69

Aruba SD-WAN Integration With Public Cloud (AWS) - Technical Note

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Aruba SD-WAN Integration With Public Cloud (AWS) - Technical Note

Uploaded by

Copyright:

Available Formats

Aruba SD-WAN Integration with

AWS Public Cloud

Open Source Code

Hewlett Packard Enterprise Company

3333 Scott Blvd

Santa Clara, CA 95054

Phone: 1-800-WIFI-LAN (+800-943-4526)

Revision History ................................................................................... 4

2 Establishment of the Overlay Network .......................................... 9

3 Single VPC deployments ............................................................... 18

4 Multi-VPC Deployments ................................................................ 39

5 Annexure—Useful AWS Procedures ............................................ 56

Aruba SD-WAN Integration with AWS Public Cloud Contents | 3

Revision Change Description

Version 1.0 Initial Publication

Table 1 Revision History

Aruba SD-WAN Integration with AWS Public Cloud Revision History | 4

1.1 vGW in Public Cloud Infrastructure

Aruba SD-WAN Integration with AWS Public Cloud Overview | 5

Figure 1 - Role of vGW

1.2 Virtual Gateway Orchestrator

Aruba SD-WAN Integration with AWS Public Cloud Overview | 6

Figure 2 - vGW Orchestrator Functions

Aruba SD-WAN Integration with AWS Public Cloud Overview | 7

Figure 3 - AWS Deployment Models

Aruba SD-WAN Integration with AWS Public Cloud Overview | 8

2.1 Aruba SD-WAN Orchestrator Overview

Figure 4 - SD-WAN Control Channel

Figure 5 - SD-WAN Tunnel Orchestration

Figure 6 - SD-WAN Route Orchestration

• Labeling WAN interfaces

2.2.1 Define WAN Uplinks

Figure 7 - Define WAN Uplinks

2.2.2 Establish SD-WAN Overlay

Figure 8 - SD-WAN DC Preference – Branch configuration

Figure 9 – SD-WAN Orchestration-DC Aggregate routes

Figure 10 –Redistribute into SD-WAN

2.3.1 Tunnel Orchestration

Figure 11 - SD-WAN Tunnel Orchestration – Connections

Figure 12 - SD-WAN Tunnel Orchestration – Tunnels

Figure 13 - SD-WAN Route Orchestration Learned/Advertised Routes

Figure 14 - Routes Learnt from SD-WAN

Figure 17 - Routing Inside a single VPC

3.1.3.1 Failover of Subnet Routes

Figure 18 - vGW HA - Initial status

Figure 19 - vGW HA - Failover

3.1.3.2 SD-WAN Failover

Figure 20 - vGW Failover - SDWAN Routing

3.2.2.1 Generate Third- Party ARN Token

Figure 21 - Generate Third-Party Token

Figure 22 - Account Details

Figure 23 - Account ID and External ID

Figure 24 - IAM Permissions

Figure 25 - IAM Role Creation

Figure 26 - Paste IAM role ARN

Figure 27 - Access to AWS Verification

Figure 28 - Create vGW

Figure 29 - vGW Deployment finished

Figure 30 - vGW Summary

3.2.4.1 Initial vGW Group Setup

Figure 31 - Group Creation

Figure 32 - Group Type

Figure 33 - Move vGW to Group